Optimal server estimate with Markov chains
I was learning about Markov chains while reading about Hidden Markov Models (HMMs), they are used in understanding and decoding the meaning behind speech. Working on system design and architecture concepts, I saw an analogy of this concept, for using them for finding an optimal number of servers for your micro-service. This post has been long overdue though, since July 16 to be precise, the day ' Amazon Prime-Day ' took place this year ; the post is inspired by that busy day for servers, where multiple customer requests hop from one server to another, sporadically sky-rocketing the read/write load on the them. For lucidity and ease of understanding, consider that there are only 5 categories and data for each category is in a separate server, and each server can serve \(100k\) customers. The most important problem for Amazon is optimizing the number of servers for each category. Let's say that through deep-learning algorithms and yada-yada we can predict the total numbe...