Generic selectors
Exact matches only
Search in title
Search in content
Post Type Selectors
Generic selectors
Exact matches only
Search in title
Search in content
Post Type Selectors

AI leadership: Learning to break free of the past

A nuanced understanding of the limitations of AI’s dependence on the past in its use of data is essential, argues Professor Alan Brown

11 March 2024

Sometimes it feels like I’m stuck in the past. Too often when faced with a new challenge, my first inclination is not to face forwards with an open mind, but to look backwards to try to extract lessons from previous experiences that help me to describe and understand it. And while relying on what’s happened before can be very helpful in many circumstances, it also brings the real danger of being too blinkered, biased, or backward. 

Unfortunately, it is also a significant concern when looking to implement and apply AI. It sees the future through the eyes of the past. Despite the futuristic allure of AI, its intrinsic strength lies in the analysis of large amounts of historical data to extrapolate future scenarios. This approach raises important questions: Is AI overly reliant on the past in steering a course through an ever-evolving strategic and operational landscape? And if so, what are the implications for how we use AI to take us forward?  

The Strengths of AI’s Backwards Looking Approach

The analytical prowess of AI, rooted in processing extensive historical data, shines a light on hidden trends, correlations, and anomalies. Consider AI’s role in enhancing many different kinds of forecasting capabilities. By scrutinizing past sales patterns, supply chain movements, customer behaviour, and market fluctuations, AI can predict future demand with an unprecedented level of accuracy, allowing optimised inventory management and personalized tailoring of marketing campaigns to individual preferences, build communities around shared products and services, and influence global trends.  

Moreover, AI’s ability to streamline operations is exemplified through its analysis of historical performance data. This allows organizations to identify operational bottlenecks, optimize production processes, and predict equipment failures, resulting in improved efficiency and reduced downtime. This underscores the transformative impact of AI on industrial operations. The innovation acceleration facilitated by AI is equally noteworthy. The mining of past research papers, patents, and industry trends enables AI to expedite the discovery of novel ideas, materials, and products. In short, well-tuned use of historical data can propel organizations and industries forward.  

The Pitfalls of Relying on the Past to Predict the Future

Yet, as we have seen all too clearly recently, predicting the future is fraught will dangers. Of course, black swan events, like pandemics or technological breakthroughs, can shatter established patterns. However, more often routine challenges pose a greater threat. Complex systems like platforms, markets, or societies are inherently dynamic, with countless factors interacting in unpredictable ways. As a result, even minor adjustments or small variations can lead to wildly divergent outcomes, making precise predictions near impossible. While data is crucial for understanding the past and present, embracing the inherent uncertainty of the future is key to navigating the uncharted waters that lie ahead.  

Consequently, a nuanced understanding of the limitations inherent in AI’s past-dependence in its use of data is essential. A clear example is the potential introduction of data bias. AI algorithms trained on skewed or outdated data risk perpetuating existing biases and inequalities. For instance, a recruitment AI system may be trained on past hiring data that embeds cultural and corporate biases concerning candidates’ background, education, ethnicity, and gender. The risk is that the AI system might inadvertently replicate this bias in future recommendations, exacerbating imbalances within the workforce.  

Another significant limitation arises from AI’s propensity to primarily extrapolate only from existing patterns. A case in point is a large language model, like ChatGPT, trained on historical news articles, which might struggle to accurately predict groundbreaking scientific discoveries or significant political upheavals due to its limited exposure to alternative possibilities beyond historical data.  

Furthermore, overreliance on AI predictions has been seen to foster a false sense of certainty among decision-makers. A balanced approach that considers alternative perspectives is crucial.  

AI’s Black Box

Underlying this challenge is often a poor understanding in leaders and decision makers of the fundamental concepts of AI and data science. Hence, many people beginning to rely on AI systems have little meaningful understanding of what’s inside the “AI black box”. A deeper scrutiny of AI’s use of data for prediction exposes several important principles that must be recognised by anyone involved with the responsible use of AI:  

When the Past Misleads: The lessons from Covid

The COVID-19 pandemic serves as an illustrative case study, demonstrating how reliance on pre-pandemic data can lead to misleading predictions. Consider the fragility of AI-supported supply chains as they struggled to cope during the pandemic. During the pandemic, AI predictions varied widely from the new business reality due to significant changes in production, sudden surges in demand, and supply chain redesigns. Unforeseen events can significantly impact markets and behaviours, and predictions based on pre-pandemic data lack context to understand such shifts. It is essential to refresh training data continuously to ensure accurate predictions. AI models rigidly reliant on past data potentially fail to adapt to changing market conditions, consumer preferences, and unforeseen disruptions. 

Breaking Free of the Past

Overcoming AI data limitation issues is far from easy. To navigate through these intricate challenges, digital leaders must adopt a strategic and proactive stance to data management, including:  

  • Embracing data diversity and remaining vigilant is paramount. AI systems need diverse and current datasets. Leaders must monitor for biases to ensure fair and accurate predictions. 
  • Human-AI collaboration is central to effective deployment of AI. AI should complement, not replace, human judgment. Combining AI’s predictive capabilities with human ingenuity is vital for tackling complex situations and exploring uncharted territories. 
  • Embracing experimentation and agility is crucial. Organizations must embrace experimentation and agile decision-making to adapt quickly to changing market dynamics. 

To use AI effectively, we need a deeper understanding of its use of historical data. While analyzing the past is important, it should not limit our future vision. By addressing the limits of AI’s reliance on historical data, we can unlock its potential and use it responsibly to guide us forward. 

This is an abridged version of an article originally posted on Alan’s digital dispatches blog. 

Start today. Change tomorrow.

If you would like to contact us to discuss potential study options, research partnerships or other collaborations then please complete the form below and we will be in touch.