Modeling and Simulation with System Dynamics: A Literature Review on Validation Aspects

Andrés Rey Piedrahita, Laura Milena Cárdenas Ardila, Jorge Andrick Parra Valencia, Ángel José Lozada Das Dores, José Gabriel Pérez Canencio

Abstract

This study aimed to address the critical issue of model validation in the field of system dynamics (SD) modeling. Methods: To achieve this objective, a comprehensive literature review was conducted to examine the existing validation approaches and practices. The analysis investigated the historical development of the validation methods, identified key trends and challenges, and explored the diverse range of qualitative, quantitative, and mixed methods employed in SD model validation. Findings: The findings revealed a significant gap between the theoretical importance of validation and its consistent practical application. The findings of this study led to the formulation of a novel conceptual framework for SD model validation. Novelty: This framework provides a structured guide for researchers to conduct more rigorous and comprehensive validation analyses that incorporate a broader range of techniques and considerations. The aim of the proposed framework is to enhance the credibility and reliability of SD models, thereby improving confidence in their use for decision-making and policy analysis.

 

Keywords: Complex systems, modeling and simulation, model validation, system dynamics, emergent behavior, validation methods, simulation analysis.

 

https://doi.org/10.55463/issn.1674-2974.52.1.3


Full Text:

PDF


References


YOON S, ANDERSON E, KLOPFER E, KOEHLER-YOM J, SHELDON J, SCHOENFELD I, … & GOH SE. Designing Computer-Supported Complex Systems Curricula for the Next Generation Science Standards in High School Science Classrooms. Systems, 2016, 4: 38. https://doi.org/10.3390/systems4040038

BARELLI E, & LEVRINI O. Computational simulations at the interface of physics and society: A teaching-learning module for high-school students. IL Nuovo Cimento. Communications: SIF Congress, 2022, 45(6): 1–4. http://dx.doi.org/10.1393/ncc/i2022-22213-6

YOON SA. An Evolutionary Approach to Harnessing Complex Systems Thinking in the Science and Technology Classroom. International Journal of Science Education, 2008, 30: 1–32. https://doi.org/10.1080/09500690601101672

HELBIG N, CRESSWELL AN, BURKE GB, & LUNA-REYES L. The dynamics of opening government data. Albany, New York: 2012.

NAJAFABADI MM, & LUNA-REYES LF. Open Government Data Ecosystems: A Closed-Loop Perspective. Proceedings of the 50th Hawaii International Conference on System Sciences, 2017, pp. 2711-2720.

ARROYO MENÉNDEZ M, & HASSAN COLLADO S. Simulación de procesos sociales basada en agentes software. Empiria Revista de Metodología de Ciencias Sociales, 2007. 14: 139-161. https://doi.org/10.5944/empiria.14.2007.1175

CASABOZA J, & CÁRDENAS DE. Analysis and Modeling of Dynamic Behavior of the COVID-19 Outbreak: Study Case of Panama. IEEE Latin America Transactions, 2021, 19: 893–900. https://doi.org/10.1109/TLA.2021.9451233

FENNER G, LIMA A, DE SOUZA J, MOURA JAB, & BEZERRA TR. Supporting Infrastructure as a Service Capacity Management through Business Scenarios Simulation. IEEE Latin America Transactions, 2020, 18: 473–486. https://latamt.ieeer9.org/index.php/transactions/article/view/210

PARHIZKAR T, & MOSLEH A. Guided Probabilistic Simulation of Complex Systems Toward Rare and Extreme Events. 2022 Annual Reliability and Maintainability Symposium (RAMS), IEEE; 2022, pp. 1–7. https://doi.org/10.1109/RAMS51457.2022.9893976

NCUBE C, & LIM SL. On Systems of Systems Engineering: A Requirements Engineering Perspective and Research Agenda. Proceedings of the 2018 IEEE 26th International Requirements Engineering Conference (RE), IEEE; 2018, pp. 112–123. https://doi.org/10.1109/RE.2018.00021

CHATTOPADHYAY S, ROY T, SENGUPTA S, & BERGER-VACHON C. Modelling and Simulation in Science, Technology and Engineering Mathematics. vol. 749. 1st ed. Cham: Springer International Publishing, 2019. https://doi.org/10.1007/978-3-319-74808-5

MALDONADO GRANADOS LF. El modelamiento matemático en la formación del ingeniero. 1st ed. Bogotá DC., Colombia: Universidad Central, 2013. https://www.ucentral.edu.co/sites/default/files/inline-files/2015_modelamiento_matematico_001.pdf.

KIM Y, & KIM J. Model validation in dynamic systems for time‐course data with complex error structures. Journal of Chemometrics, 2019, 33(5): e3108. https://doi.org/10.1002/cem.3108

GARCÍA-VALDECASAS MEDINA JI. Explicación, mecanismo y simulación: otra manera de hacer sociología. Empiria Revista de Metodología de Ciencias Sociales, 2014, 28: 35. https://doi.org/10.5944/empiria.28.2014.12120

BARLAS Y, & CARPENTER S. Philosophical roots of model validation: Two paradigms. System Dynamics Review, 1990, 6(2): 148–166. https://doi.org/10.1002/sdr.4260060203

BARLAS Y. Formal aspects of model validity and validation in system dynamics. System Dynamics Review, 1996, 12: 183–210. https://doi.org/10.1002/(SICI)1099-1727(199623)12:3%3C183::AID-SDR103%3E3.0.CO;2-4

SCHOENBERG W, & SWARTZ J. System Dynamics, Machine Learning and Structural Validation. LIFE: A Transdisciplinary Inquiry, Intellect Books/University of Chicago Press, 2024, pp. 289–308.

BARLAS Y. Multiple tests for validation of system dynamics type of simulation models. European Journal of Operational Research, 1989, 42: 59–87. https://doi.org/10.1016/0377-2217(89)90059-3

BALCI O. Principles and techniques of simulation validation, verification, and testing. 1995 Winter Simulation Conference Proceedings, IEEE, 1995, pp. 147–154. https://doi.org/10.1109/WSC.1995.478717

BALCI O. Validation, verification, and testing techniques throughout the life cycle of a simulation study. Annals of Operational Research, 1994, 53: 121–173. https://doi.org/10.1007/BF02136828

XIANG X, KENNEDY RC, MADEY GR, & CABANISS SE. Verification and Validation of Agent-based Scientific Simulation Models. Proceedings of the Agent-Directed Simulation Conference, 2005, pp. 47-55. https://www3.nd.edu/~nom/Papers/ADS019_Xiang.pdf.

SCHOLL HJ (JOCHEN), & LUNA-REYES LF. Transparency and openness in government: a system dynamics perspective. Proceedings of the 5th International Conference on Theory and Practice of Electronic Governance, New York, NY, USA: Association for Computing Machinery; 2011, pp. 107–114. https://doi.org/10.1145/2072069.2072088

LUTZ CS, CARR W, COHN A, & RODRIGUEZ L. Understanding barriers and predictors of maternal immunization: Identifying gaps through an exploratory literature review. Vaccine, 2018, 36: 7445–7455. https://doi.org/10.1016/j.vaccine.2018.10.046

NAUGLE A, LANGARUDI S, & CLANCY T. What is (quantitative) system dynamics modeling? Defining characteristics and the opportunities they create. System Dynamics Review, 2024, 40(2): e1762. https://doi.org/10.1002/sdr.1762

COYLE G, & EXELBY D. The validation of commercial system dynamics models. System Dynamics Review, 2000, 16: 27–41. https://doi.org/10.1002/(SICI)1099-1727(200021)16:1<27::AID-SDR182>3.0.CO;2-1

QUDRAT-ULLAH H. Structural validation of system dynamics and agent-based simulation models. Simulation in Wider Europe - 19th European Conference on Modelling and Simulation, ECMS, 2005.

https://scs-europe.net/services/ecms2005/pdf/abs-07.pdf.

HOMER JB. Partial‐model testing as a validation tool for system dynamics (1983). System Dynamics Review, 2012, 28: 281–294. https://doi.org/10.1002/sdr.1478

QUDRAT-ULLAH H. On the validation of system dynamics type simulation models. Telecommunication Systems, 2012, 51: 159–166. https://doi.org/10.1007/s11235-011-9425-4

VAN NISTELROOIJ LPJ, ROUWETTE EAJA, VERSTIJNEN IM, & VENNIX JAM. The Eye of the Beholder: A Case Example of Changing Clients’ Perspectives Through Involvement in the Model Validation Process. Systems Research and Behavioral Science, 2015, 32: 437–449. https://doi.org/10.1002/sres.2336

LAIMON M, MAI T, GOH S, & YUSAF T. System dynamics modelling to assess the impact of renewable energy systems and energy efficiency on the performance of the energy sector. Renewable Energy, 2022, 193: 1041–1048. https://doi.org/10.1016/j.renene.2022.05.041

KANNAN U, & SWAMIDURAI R. Empirical Validation of System Dynamics Cyber Security Models. Proceedings of the 2019 SoutheastCon, IEEE, 2019, pp. 1–6. https://doi.org/10.1109/SoutheastCon42311.2019.9020607

HUANG H, CHEN W, XIE T, WEI Y, FENG Z, & WU W. The Impact of Individual Behaviors and Governmental Guidance Measures on Pandemic-Triggered Public Sentiment Based on System Dynamics and Cross-Validation. International Journal of Environmental Research and Public Health, 2021, 18: 4245. https://doi.org/10.3390/ijerph18084245

JIA S., LI Y. & FANG T. System dynamics analysis of COVID-19 prevention and control strategies. Environmental Science and Pollution Research, 2022, 29: 3944–3957. https://doi.org/10.1007/s11356-021-15902-2

SHALAHUDDIN M, SUNINDYO WD, EFFENDI MR, & SURENDRO K. Fuzzy‐set qualitative comparative analysis for validating causal relationships in system dynamics models. Engineering Reports, 2024, 6: e12855. https://doi.org/10.1002/eng2.12855

LI Q, WANG Z, LI L, HAO H, CHEN W, & SHAO Y. Machine learning prediction of structural dynamic responses using graph neural networks. Computers and Structures, 2023, 289: 107188. https://doi.org/10.1016/j.compstruc.2023.107188

FORRESTER JW. System dynamics, systems thinking, and soft OR. System Dynamics Review, 1994, 10: 245–256. https://doi.org/10.1002/sdr.4260100211

STERMAN J. System Dynamics: Systems Thinking and Modeling for a Complex World. Cambridge MA: 2002. https://archive.org/details/businessdynamics0000ster

BELETE GF, VOINOV A, & LANIAK GF. An overview of the model integration process: From pre-integration assessment to testing. Environmental Modelling & Software, 2017, 87: 49–63. https://doi.org/10.1016/j.envsoft.2016.10.013

SCHRUBEN LW. Establishing the credibility of simulations. Simulation, 1980, 34: 101–105. https://doi.org/10.1177/003754978003400310

TEDESCHI LO. Assessment of the adequacy of mathematical models. Agricultural Systems, 2006, 89(1-2): 225–247. https://doi.org/10.1016/j.agsy.2005.11.004

ROBINSON S, & BROOKS RJ. Independent Verification and Validation of an Industrial Simulation Model. Simulation, 2010, 86: 405–416. https://doi.org/10.1177/0037549709341582

PACE DK. Verification, validation, and accreditation of simulation models. In: OBAIDAT, M.S., & PAPADIMITRIOU, G.I. (eds) Applied System Simulation. Applied System Simulation: Methodologies and Applications, Springer, Boston, MA.: Kluwer Academic Publishers; 2003, chapter 21: 487–506.


Refbacks

  • There are currently no refbacks.