What a Simulation Can Tell Us
Suppose a computer-generated trajectory passes almost exactly through a set of measured points. Has it confirmed a physical theory? Has it shown that the software works? Or has a parameter merely been adjusted until the graph looks right?
Each is a different question. A simulation can help us explore systems that resist a simple calculation, test sensitivity to assumptions, and decide what to measure. Its power comes from making a model’s consequences available for inspection. The inspection becomes scientific when we can say exactly which result was checked, against what, and under which conditions.[1]
In Quantum Made Clear, the previous article, we separated preparation, prediction, observation, and interpretation. This topic adds a step often hidden between prediction and observation: a numerical computation. That step can reveal structure, but it can also introduce error.
A computation has a target
Begin with a question. For a pendulum, we might want its angle after two seconds, the time of a swing or its sensitivity to initial angle. Each is a quantity of interest: the specific output on which a conclusion depends.[1]
Next comes a physical and mathematical model. At small angles, a pendulum can be approximated by an equation relating its angular acceleration to its angle. To describe damping, we may add a term proportional to angular velocity. The equation represents assumptions about the pivot, air resistance, geometry, and the size of the swing. It does not contain every feature of a real device.
A numerical method then advances an approximate solution through small time steps. Changing the step size is a way to investigate numerical error. If a reported output changes substantially as the step is refined, the original calculation was not yet stable enough to support that output. If it settles, the numerical solution may be converging to the chosen equation’s solution. It has not thereby shown that the equation captures a large-angle pendulum.[1,3]
This is simulation’s distinctive contribution: we can intervene on the model, vary its inputs, expose sensitivities, and derive conditional predictions even when an exact formula is difficult. Calling it a “third mode” alongside theory and experiment is a useful description of scientific practice. Those modes cooperate: the model supplies structure, computation explores it, and physical observations constrain the connection to nature.
Four checks answer four questions
Code verification: Did the software implement the intended method? Tests against known solutions and controlled benchmark problems help locate implementation mistakes. Solution verification: How much numerical error remains in the output of this run? Refinement studies and error estimates address that question. These two tasks concern the calculation.[1,3]
Validation: How well does the modeled system represent the physical referent for the intended use? Relevant measurements, their uncertainty, and the tested range matter. NASA’s modeling standard explicitly ties validation to intended uses and records a domain of validation; it also distinguishes conceptual from empirical validation.[2] A model can pass a useful test for modest swings yet lack support for large swings or a different apparatus.
Calibration: What parameter values fit selected observations? Estimating the damping coefficient from a recorded trajectory can be valuable. But comparing the adjusted model only to that same trajectory does not independently test its predictive performance. Statistical work on computer-model calibration also recognizes that even the best parameter values may leave structural discrepancy between a model and the world.[5]
The four checks are related, not interchangeable. Verification can succeed while the physical assumptions fail. Calibration can succeed because adjustable parameters conceal a missing mechanism. Validation of one output under one condition does not certify every output under every condition.[1,2,7]
Agreement has a scope
Imagine two simulations of a moving robot. One assumes constant tire friction and no sensor delay. Another includes changing friction and delayed feedback. Both might reproduce a short run on a clean floor after tuning. On loose ground, their predicted stopping distances could diverge. This is a hypothetical example, not evidence about a particular robot.
To evaluate the models, first specify the decision: for example, is the estimated stopping distance adequate for a defined surface, load, and speed? Identify which observations set parameters and reserve relevant new conditions for comparison. Examine uncertainty in friction, delay, and measurements. Then test whether the decision changes across plausible values and model structures. Where safety matters, the required credibility should reflect how much the decision relies on the model and the consequences of error; ASME articulates that principle in its risk-based medical-device modeling standard.[6]
There is a legitimate counterargument to demanding a physical test for every situation. Some conditions are inaccessible, dangerous, or too costly to recreate. Simulation can then be the best available way to examine scenarios and choose a more informative experiment. The appropriate conclusion remains conditional: which assumptions were tested, what uncertainty was assessed, and how far the proposed use extends beyond observed conditions?[1,2]
Oreskes and colleagues warned that matching a natural system cannot establish a numerical model as uniquely true, particularly when systems are open and alternative models can reproduce observations.[7] Engineering validation uses a narrower operational question: how accurately does this model serve this intended use? These positions can coexist when “validated” is reported with a quantity, criterion, range, and purpose rather than treated as a universal certificate.[1,2,7]
A better scientific question
Before accepting a compelling simulation, ask: What equation or rule was encoded? Which numerical output was checked? Which observations were independent of fitting? Under what conditions would this model fail? A good report lets someone else rerun the calculation and challenge the physical inference.
For a classroom, one can compare a simulated small-angle pendulum with a measured larger swing. The point is to have learners explain a discrepancy and choose the next test. The educational value of that proposed activity has not been measured here. For a research or engineering team, the same discipline makes a decision more accountable.
Simulation deserves its place in scientific inquiry precisely because it can make assumptions consequential and visible. A close fit is a beginning of investigation. The scientific result is a carefully delimited claim about what the computation and the observations together support.
Articles related: Applied and Theoretical Physics as One Scientific Cycle; Models, Measurements and the Limits of Prediction; And Quantum Made Clear.
References
- National Research Council. (2012). Assessing the reliability of complex models: Mathematical and statistical foundations of verification, validation, and uncertainty quantification. National Academies Press. https://doi.org/10.17226/13395
- National Aeronautics and Space Administration. (2024). Standard for models and simulations (NASA-STD-7009B). https://standards.nasa.gov/sites/default/files/standards/NASA/B/1/NASA-STD-7009B-Final-3-5-2024.pdf
- Oberkampf, W. L., & Trucano, T. G. (2002). Verification and validation in computational fluid dynamics. Progress in Aerospace Sciences, 38(3), 209–272. https://doi.org/10.1016/S0376-0421(02)00005-2
- American Society of Mechanical Engineers. (n.d.). Verification, validation and uncertainty quantification (VVUQ). https://www.asme.org/codes-standards/publications-information/verification-validation-uncertainty
- Kennedy, M. C., & O’Hagan, A. (2001). Bayesian calibration of computer models. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 63(3), 425–464. https://doi.org/10.1111/1467-9868.00294
- American Society of Mechanical Engineers. (2018). Assessing credibility of computational modeling through verification and validation: Application to medical devices (ASME V&V 40–2018). https://www.asme.org/codes-standards/find-codes-standards/assessing-credibility-of-computational-modeling-through-verification-and-validation-application-to-medical-devices
- Oreskes, N., Shrader-Frechette, K., & Belitz, K. (1994). Verification, validation, and confirmation of numerical models in the earth sciences. Science, 263(5147), 641–646. https://doi.org/10.1126/science.263.5147.641