Papers & Presentations

You may request a copy of any paper without a direct link. We've organized things into categories for different audiences:

• Introductory materials, providing an overview of the philosophy and approaches we use when designing and analyzing simulation experiments;
• Selected articles and book chapters that describe methodological advances;
• Selected proceedings papers that contain a mix of theory and applications.

Videos

Work Smarter, Not Harder: A Tutorial on Designing and Conducting Simulation Experiments video by Susan Sanchez, Paul Sanchez, and Hong Wan (North Carolina State University) presented Dec 14, 2021 at the 2021 Winter Simulation Conference, Phoenix, AZ. WARNING - THIS VIDEO IS 340 MB and 1 hour 12 minutes long.

WORMS and Work in the Time of Coronavirus, webinar by Susan Sanchez, presented May 15, 2020.  WARNING - THIS VIDEO IS 500 MB and 60 minutes long.  Click here for slides.

A Data Farmer's Almanac, "Titan Talk" video by Susan Sanchez, presented December 13, 2016 at the 2016 Winter Simulation Conference, Washington, DC.  WARNING - THIS VIDEO IS 527 MB and 60 minutes long.

Introduction to Data Farming video by Paul Sanchez, presented May 8, 2010 at CENIC Conference, NPS, Monterey, CA. WARNING - THIS VIDEO IS 336 MB and 35 minutes long.

Introductory Papers

Sanchez, S. M. (2018). Data farming: reaping insights from simulation models. Chance, 2(31), 45-52.

Lucas, T.W., W.D. Kelton, P.J. Sanchez, S.M. Sanchez, and B.L. Anderson. 2015. Changing the Paradigm: Simulation, Now a Method of First Resort. Naval Research Logistics 62: 293–305.

Kleijnen, J.P.C., S.M. Sanchez, T.W. Lucas, and T.M. Cioppa. 2005. A user's guide to the brave new world of designing simulation experiments. INFORMS Journal on Computing 17(3), 263–289 (with online supplement).

Sanchez, S. M., P.J. Sanchez, and H. Wan (2018). Work smarter, not harder: A tutorial on designing and conducting simulation experiments. Proceedings of the 2018 Winter Simulation Conference, 237-251. Updated from earlier tutorials that appeared in 2015, 2012, 2011, 2009, ...,

Sanchez, S. M. (2015). "Simulation experiments: Better data, not just big data." Proceedings of the 2015 Winter Simulation Conference, 800-811. (Invited, expanded tutorial version of 2014 paper)

Lucas, T.W., S.M. Sanchez, F. Martinez, J.W. Roginski, L.R. Sickinger. 2007. Defense and homeland security applications of multi-agent simulations. Proceedings of the 2007 Winter Simulation Conference, 138–149.

Sanchez, S.M., T.W. Lucas, P.J. Sanchez, C.J. Nannini, and H. Wan. 2012. Designing large scale simulation experiments, with applications to defense and homeland security. Chapter 12 in Design and Analysis of Experiments, Vol. 3: Special Designs and Applications, ed. K. Hinkelmann. Hoboken, NJ: Wiley.Winner, 2013 Koopman Prize, INFORMS Military Applications Society.

More on Experimental Design and Analysis

Sanchez, P. J., & Sanchez, S. M. (2018). Orthogonal 2nd order space-filling designs with insights from simulation experiments to support test planning. Quality and Reliability Engineering International, 35(3), 854-867.

Erickson, C., Ankenman, B. E., & Sanchez, S. M. (2018). Comparison of Gaussian process modeling software. European Journal of Operational Research, 2018(266), 179-192.

Duan, W., B. E. Ankenman, P. J. Sanchez, and S. M. Sanchez (2017). Sliced full factorial-based Latin hypercube designs as a framework for a batch sequential design algorithm." Technometrics, forthcoming.  Available prior to print edition in: Taylor & Francis Online, http://www.tandfonline.com/doi/full/10.1080/00401706.2015.1108233

Erickson, C., B. E. Ankenman, and S. M. Sanchez (2016). Comparison of Gaussian process modeling software. Extended abstract for poster session, Proceedings of the 2016 Winter Simulation Conference

MacCalman, A. D., H. Vieira, and T. W. Lucas (2016). Second-order nearly orthogonal Latin hypercubes for exploring stochastic simulations.  Journal of Simulation, available online at http://link.springer.com/article/10.1057/jos.2016.8.

Sanchez, S. M. (2015). "Simulation experiments: Better data, not just big data." Proceedings of the 2015 Winter Simulation Conference, 800-811. (Invited, expanded tutorial version of 2014 paper)

Sanchez, S. M. and H. Wan (2015). Work smarter, not harder: A tutorial on designing and conducting simulation experiments. Proceedings of the 2015 Winter Simulation Conference, 1795-1809. Updated from earlier tutorials that appeared in 2012, 2011, 2009, ...,

Sanchez, S. M. (2014). "Simulation experiments: Better data, not just big data." Proceedings of the 2014 Winter Simulation Conference, 805-806. (Invited, featured speaker in "Big Data" track)

Vieira Jr., H., S. M. Sanchez, P. J. Sanchez, K. H. Kienitz, M. C. M. Belderrain (2014). “A restricted multinomial hybrid selection procedure,” ACM Transactions on Modeling and Computer Simulation, 24(2), Article 10, pp. 10:1–10:24.

Sanchez, S. M., P. J. Sanchez, and H. Wan (2014). Simulation experiments: Better insights by design. Proceedings of the 2014 Summer Simulation Multi-Conference.

"Data farming in support of NATO." NATO Science & Technology Organization Technical Report TR-MSG-088. Available via https://www.cso.nato.int/Pubs/rdp.asp?RDP=STO-TR-MSG-088. 226 pages. Winner, 2014 Scientific Achievement Award, NATO STO.

Vieira Jr., H., S. M. Sanchez, K. H. Kienitz, M. C. M. Belderrain (2013). “Efficient, nearly orthogonal-and-balanced, mixed designs: An effective way to conduct trade-off analyses via simulation,” Journal of Simulation 7: 264–275 (Special Issue on Input/Output Analysis).

Hernandez, A., Lucas, T. W., and Carlyle, M. (2012). "Constructing nearly orthogonal Latin hypercubes for any nonsaturated run-variable combination." ACM Trans. Model. Comput. Simul. 22, 4, Article 20 (November 2012)

Sanchez, S. M. and H. Wan (2012). Work smarter, not harder: A tutorial on designing and conducting simulation experiments. " Proceedings of the 2012 Winter Simulation Conference, eds. C. Laroque, J. Himmelspach, R. Pasupathy, O. Rose, and A. M. Uhrmacher, Piscataway, New Jersey: Institute of Electrical and Electronic Engineers, Inc., 1552–1563. (Invited). Updated from earlier tutorials that appeared in 2011, 2009, ...,

Hernandez, A., Lucas, T. W. and Sanchez, P. J. (2012). Selecting Random Latin Hypercube Dimensions And Designs Through Estimation Of Maximum Absolute Pairwise Correlation. Proceedings of the 2012 Winter Simulation Conference, eds. C. Laroque, J. Himmelspach, R. Pasupathy, O. Rose, and A. M. Uhrmacher, Piscataway, New Jersey: Institute of Electrical and Electronic Engineers, Inc.

Vieira Jr., H., S. Sanchez, K. H. Kienitz, and M. C. N. Belderrain (2011). “Generating and improving orthogonal designs by using mixed integer programming,” European Journal of Operational Research 215: 629–638.

Sanchez, P. J., 2011, "Efficient generation of resolution VII fractional factorial designs," notes.

Sanchez, S. M. and H. Wan (2011). Better than a petaflop: The power of efficient experimental design. Proceedings of the 2011 Winter Simulation Conference, 1441-1455.

Schruben, L. and D. Singham (2011).  Agent based simulation output analysis.  Proceedings of the 2011 Winter Simulation Conference, 540-548.

Vieira Jr., H., S.M. Sanchez, K.H. Kienitz, and M.C.N. Belderrain (2011). Improved efficient, nearly orthogonal, nearly balanced mixed designs. Proceedings of the 2011 Winter Simulation Conference, 3605-3616.

Vieira Jr., H., S. Sanchez, K.H. Kienitz, and M.C.N. Belderrain. 2011. Generating and improving orthogonal designs by using mixed integer programming. European Journal of Operational Research 215: 629–638.

Shen, H., H. Wan, and S.M. Sanchez. 2010. A hybrid method for simulation factor screening. Naval Research Logistics 57(1):  45–57.

Oh, R.P.T., S.M. Sanchez, H. Wan, M.E. Nissen, and T.W. Lucas, 2009, "Efficient experimental design tools for exploring large simulation models," Computational & Mathematical Organizational Theory 15(3): 237–257.

Sanchez, S.M., H. Wan, and T.W. Lucas. 2009. Two-phase screening procedure for simulation experiments. ACM Transactions on Modeling and Computer Simulation 19(2): 1–24. DOI= http://doi.acm.org/10.1145/1502787.1502790

Cioppa, T.M., and T.W. Lucas. 2007. Efficient nearly orthogonal and space-filling Latin hypercubes. Technometrics 49(1): 45–55.

Sanchez, S. M., F. Moeeni, and P. J. Sanchez. 2006. So many factors, so little time...simulation experiments in the frequency domain. International Journal of Production Economics, 103(1): 149–165.

Sanchez, S. M. and P. J. Sanchez. 2005. Very large fractional factorials and central composite designs. ACM Transactions on Modeling and Computer Simulation 15(4): 362–377.

Vinyard, W. and T. W. Lucas. 2002. Exploring combat models for non-monotonicities and remedies. PHALANX 35(1): 19, 36–38.

General Simulation Analysis Topics

Lucas, T.W., W.D. Kelton, P.J. Sanchez, S.M. Sanchez, and B.L. Anderson. 2015. Changing the Paradigm: Simulation, Now a Method of First Resort. Naval Research Logistics 62: 293–305.

Elmegreen, B. E., S. M. Sanchez, and A. Szalay. (2014). The future of computerized decision making. Proceedings of the 2014 Winter Simulation Conference, 943-949.

Powers, M. J., S. M. Sanchez, and T. W. Lucas (2012). The exponential expansion of simulation in research. Proceedings of the 2012 Winter Simulation Conference, 1929–1943.

Singham, D. and L.W. Schruben. 2012. Finite-sample performance of absolute precision stopping rules. INFORMS Journal on Computing, 24(4): 624-635.

Sanchez, P.J. and K.P. White. 2011. Interval estimation using replication/deletion and MSER truncation. Proceedings of the 2011 Winter Simulation Conference, 488-494.

More on Modeling Methodology

Sanchez, P. J. and S. M. Sanchez (2015).  A scalable discrete event stochastic agent-based model of infectious disease propagation.  Proceedings of the 2015 Winter Simulation Conference, 151-158.

Singham, D. and M. Thompson. 2011. Application of flocking algorithms to input modeling for agent movement. Proceedings of the 2011 Winter Simulation Conference, 2443-2449.

Buss, A.H. and A. Al Rowaei. 2010. A comparison of the accuracy of discrete event and discrete time. Proceedings of the 2010 Winter Simulation Conference, 1468–1477.

Sanchez, P. J. 2006. "As simple as possible, but no simpler: A gentle introduction to simulation modeling," Proceedings of the 2006 Winter Simulation Conference, 1–10.

Buss, A. H. and P. J. Sanchez. 2005. "Simple movement and detection in discrete event simulation," Proceedings of the 2005 Winter Simulation Conference, 992–1000.

More on Applications

Marlow, D., Sanchez, S. M., & Sanchez, P. J. (2019). Testing policies and key influences on long-term aircraft fleet management using designed simulation experiments. Military Operations Research, Vol 24 (3), 5 -25.

Kelser, G., T.W. Lucas, and P.J. Sanchez (2019), "A Data Farming Analysis of a Simulation of Armstrong's Stochastic Salvo Model," forthcoming in Proceedings of the 2019 Winter Simulation Conference, Institute of Electrical and Electronic Engineers: Piscataway, NJ, 2019.

Morgan, B. L., H. C. Schramm, J. R. Smith, T. W. Lucas, M. L. McDonald, P. J. Sanchez, S. M. Sanchez, and S. C. Upton (2017). Improving U.S. Navy campaign analysis using big data. Special Issue on Applications of Analytics and Operations Research in Big Data Analysis, Interfaces. Available online at https://doi.org/10.1287/inte.2017.0900

MacCalman, A. D., Sanchez, S. M., M. L. McDonald, S. R. Goerger, and A. T. Karl (2016). Tradespace analysis for multiple performance measures. Proceedings of the 2016 Winter Simulation Conference, 3063-3074.

Marlow, D. O., S. M. Sanchez, and P. J. Sanchez (2015). Testing aircraft fleet management policies using designed simulation experiments. Proceedings of the 21st International Congress on Modelling and Simulation, 917--923. Available online at www.mssanz.org.au/modsim2015. Winner, 2015 Gus Schaefer Award, for the Best Paper at the Defence Operations Research Symposium (DORS), held concurrently with MODSIM2015, Gold Coast, Australia.

McDonald, M. L., S. C. Upton, C. N. Seymour, T. W. Lucas, S. M. Sanchez, P. J. Sanchez, H. C. Schramm, J. R. Smith (2014). Enhancing the Analytic Utility of the Synthetic Theater Operations Research Model (STORM). Proceedings of the 2014 Winter Simulation Conference, 4136-4137.

Xing, D., H. Wan, M.Y. Zhu, S. M. Sanchez, and T. Kaymal (2013). "Simulation screening experiments using Lasso-optimal supersaturated design and analysis: A maritime operations application." Proceedings of the 2013 Winter Simulation Conference, Piscataway, New Jersey: Institute of Electrical and Electronic Engineers, Inc., 497–508.

Kang, K., and M. L. McDonald. 2010. Impact of logistics on readiness and life cycle cost. Proceedings of the 2010 Winter Simulation Conference, 1336–1346.

Escher, L., S. Hall, E. Regnier, P. J. Sanchez, and J. Hansen. 2010. Simulating pirate behavior to exploit environmental information. Proceedings of the 2010 Winter Simulation Conference, 1330–1335.

Horne, G. E. and T. Meyer, 2010, Data farming and defense applications. MODSIM World Conference and Expo, "21st Century Decision-Making, The Art of Modeling & Simulation", Hampton Roads Convention Center, Hampton, VA, USA 13–15 October 2010

Ahner, D. K., A. H. Buss, and J. Ruck. 2006. "Assignment scheduling capability for unmanned aerial vehicles - a discrete event simulation with optimization in the loop approach to solving a scheduling problem," Proceedings of the 2006 Winter Simulation Conference, 1349–1356.

Buss, A. H. and D. K. Ahner. 2006. "Dynamic allocation of fires and sensors (DAFS): A low-resolution simulation for rapid modeling," Proceedings of the 2006 Winter Simulation Conference, 1357–1364.

Kang, K., K. H. Doerr, and S. M. Sanchez. 2006. "A design of experiments approach to readiness risk analysis," Proceedings of the 2006 Winter Simulation Conference, 1332–1339.

Sanchez, S. M. 2006. "Work smarter, not harder: Guidelines for designing simulation experiments," Proceedings of the 2006 Winter Simulation Conference, 47–57.

Lucas, T. W. and S. M. Sanchez. 2006. "The brave new world of designing simulation experiments for defense and homeland security applications," 2006 JSM Proceedings.

Forsyth, A. J., S. M. Sanchez, H. Wan, K. M. Chang, and P. J. Sanchez. 2006. "Exploring edge organizations for network centric operations," Proceedings of the 2006 CCRTS Conference.

Sanchez, S. M., H. Wan, and T. W. Lucas. 2006. "A two-phase screening procedure for simulation experiments" Proceedings of the 2005 Winter Simulation Conference, 223–230.

Sanchez, S. M. 2005. "Work smarter, not harder: Guidelines for designing simulation experiments," Proceedings of the 2005 Winter Simulation Conference, 69–82.

Cioppa, T. M., T. W. Lucas, and S. M. Sanchez. 2004. "Military applications of agent-based simulations," Proceedings of the 2004 Winter Simulation Conference, 171–180.

Allen, T. E., A. H. Buss and S. M. Sanchez. 2004. "Assessing obstacle location accuracy in the REMUS unmanned underwater vehicle," Proceedings of the 2004 Winter Simulation Conference, 940–948.

Sanchez, S. M. and H.-F. Wu. 2003. "Frequency-based designs for terminating simulation experiments: A peace-enforcement example," Proceedings of the 2003 Winter Simulation Conference, 952–959.

Lucas, T. W., S. M. Sanchez, T. M. Cioppa, and A. I. Ipekci. 2003. "Generating hypotheses on fighting the global war on terrorism," Maneuver Warfare Science 2003, Marine Corps Combat Development Command, 2003, 117–137.

Sanchez, S. M. and T. W. Lucas. 2002. "Agent-based simulations: Simple models, complex analyses," Proceedings of the 2002 Winter Simulation Conference, 116–126.

Lucas, T. W., S. M. Sanchez, L. P. Brown, and W. Vinyard. 2002. "Better designs for high-dimensional explorations of distillations," Maneuver Warfare Science 2002, Marine Corps Combat Development Command, 17–46.