Problem Statement
Despite the recognized potential of AI/ML in demand planning, many organizations struggle with its effective adoption, often due to a lack of clarity regarding the optimal balance between automation and human intervention. This uncertainty is compounded by issues like algorithm aversion, which stems from a distrust of ‘black-box’ AI systems. The core problem is the absence of a defined framework to reliably determine when and how human input adds value to AI forecasts. This study addresses this by empirically distinguishing and quantifying the effects of three collaboration types—automation, adjustable automation, and augmentation—across 1,900 SKUs in demand planning. By linking these collaboration types to product characteristics, the research aims to provide a strategic, evidence-based roadmap for managerial practice, thereby increasing organizational trust and improving the overall efficacy of Human-AI collaborative demand prediction systems.
Research Question
How and under what conditions does human intervention enhance AI/ML-driven demand forecasting accuracy and inventory levels compared to full automation?
Research Methodology
The study relies on a real field experiment in the retail industry involving AI/ML-driven demand forecasts for approximately 1,900 stock-keeping units. The analysis uses a multivalued treatment effect methodology to measure the causal effects of the different Human-AI collaboration types on forecast accuracy and demand planning. Additional methods include Causal Inference and Mediation-moderation Analysis.
Key Results
The study shows when and how human input enhances AI forecasting by defining the optimal mix of automation and expertise based on product characteristics. Key findings include:
- Human Input Value: The research proposes a framework outlining the conditions in which human intervention is most likely to add predictive value to Human-AI collaborations.
- Adoption Friction: Our findings show a reluctance among demand planners to rely on AI-driven algorithms in unpredictable demand environments.
- Implications: The work provides implications for theory and enhances contextualization for effective AI adoption in managerial practice.
References
Revilla E., Saenz M.J., Namdar, J. and Seifert M. The Double-Edged Sword of Augmented Forecasting in Demand Planning. (Work in progress)
Saenz M.J., Revilla E., and Simon C. (March 2020). Designing AI Systems with Human-Machine Teams. MIT Sloan Management Review.