An Integrated Simulation, Learning and Game-theoretic Framework for Supply Chain Competition

Abstract

An integrated simulation, learning, and game-theoretic framework is proposed to address the dynamics of supply chain competition. The proposed framework is composed of 1) simulation-based game platform, 2) game solving and analysis module, and 3) multi-agent reinforcement learning module. The simulation-based game platform supports multi-paradigm modeling, such as agent-based modeling, discrete-event simulation, and system dynamics modeling. The game solving and analysis module is designed to include various parts including strategy refinement, data sampling, game solving, equilibrium conditions, solution evaluation, as well as comparative statistics under varying parameter values. The learning module facilitates the decision making of each supply chain competitor under the stochastic and uncertain environments considering different learning strategies. The proposed integrated framework is illustrated for a supply chain system under the newsvendor problem setting in several phases. At phase 1, an extended newsvendor competition considering both the product sale price and service level under an uncertain demand is studied. Assuming that each retailer has the full knowledge of the other retailer's decision space and profit function, we derived the existence and uniqueness conditions of a pure strategy Nash equilibrium with respect to the price and service dominance under additive and multiplicative demand forms. Furthermore, we compared the bounds and obtained various managerial insights. At phase 2, to extend the number of decision variables and enrich the payoff function of the problem considered at phase 1, a hybrid simulation-based framework involving systems dynamics and agent-based modeling is presented, followed by a novel game solving procedure, where the procedural components include strategy refinement, data sampling, gaming solving, and performance evaluation. Various numerical analyses based on the proposed procedure are presented, such as equilibrium accuracy, quality, and asymptotic/marginal stability. At phase 3, multi-agent reinforcement learning technique is employed for the competition scenarios under a partial/incomplete information setting, where each retailer can only observe the opponent' behaviors and adapt to them. Under such a setting, we studied different learning policies and learning rates with different decay patterns between the two competitors. Furthermore, the convergence issues are discussed as well. Finally, the best learning strategies under different problem scenarios are devised.