Although the advent of generative AI challenges the basics of business education, many experts believe it will help academics push students further, writes Alex Katsomitros
Few students pass “Operations Management”, a typical MBA course, with an A+, and even the most diligent of students at the Wharton School, University of Pennsylvania, one of the world’s most prestigious business schools, would be happy with a humble grade B. So Christian Terwiesch, an academic who teaches the subject at Wharton, was perplexed when a student answered most questions for the course’s final exam with immaculate reasoning and even “elegant” writing worthy of the grade, and yet made mistakes in simple calculations that betrayed a weak grasp of mathematics.
The problem? The student in question was not a human, but an AI-powered chatbot named ChatGPT. Its much-discussed launch last November piqued the interest of the German academic who, as an expert on operations, understood well the immense capabilities of AI. Curious to see how the chatbot would cope with a task that requires analytical skills, Terwiesch fed it exam questions that required rigorous understanding of the subject. Surprisingly enough, ChatGPT provided thoughtful and mostly accurate answers, even correcting its errors in response to simple hints; its lack of mathematical nous was the main snag. “I have used language processing in the past as a research tool, but not with wordy questions like those I fed it with,” says Terwiesch, adding: “I didn’t expect a half-page text as a response to the prompt. The fact that it sliced through all that material so elegantly and came up with a right answer, and also the perfect wording, was a surprise.”
Academics rage against the machine
Developed by the California-based AI research lab OpenAI, ChatGPT epitomises the rapid evolution of “generative” AI, which focuses on content generation through machine-learning models called “logic learning machines” (LLMs). Given the right prompt, LLMs can produce essays, novels, software and even research papers. The creative capacity of ChatGPT is nearly boundless. When asked to provide an analysis of Disney’s Bambi through the gritty lens of Martin Scorsese, it delivered an elegant, if slightly inaccurate prose: “In Scorsese’s hands, Bambi becomes a visceral and gripping tale of survival and redemption, exploring the dark underbelly of the forest and the harsh realities of life and death.” It is such eloquence that has made it the fastest growing consumer app in history, entertaining over 100 million users with its musings, just two months after its launch. Its success has pushed other technology companies to launch their own chatbots, including Google’s Bard and Meta’s LLaMA.
One reason this digital know-it-all has caused a stir is its potential impact on knowledge workers. It is one thing for robots to automate factory assembly lines; carrying out tasks that require analytical skills, hitherto a privilege of well-paid white-collar workers, is both unprecedented and ominous, spelling trouble for lawyers, accountants, journalists and even designers and architects. Business schools sense the coming turbulence too. The core skill business education offers is mastering the art of decision-making through the study of real-world case studies and decision theory. Automating this deeply human skill could reduce the value of MBA programmes. As Terwiesch notes in a paper he authored about his experience with ChatGPT: “Its impact on business school education goes beyond what its impact will be on teaching mathematics, history, biology, or literature because it affects the way people work, now and in the future.”
Inevitably, changes start with curricula. If machines are already making decisions, why not focus on human-AI collaboration and train executives on how to tap into their strengths and recognise their weaknesses? “Using AI in executive education has an added benefit in bringing the technology to life for executives who are going to have to use it, and decide on AI investments more in the future as it becomes more mainstream,” says Kelly Palmer, a former LinkedIn executive and current Chief Learning Officer at Degreed, an edtech company. Some think that AI tools can serve as digital consultants or sparring partners that can rapidly generate various solutions to a problem. Executive education needs to integrate AI into curricula to prepare managers for a bot-driven economy, argues Bernard Marr, a futurist and author of Future Skills: The 20 Skills and Competencies Everyone Needs to Succeed in a Digital World, including skills such as data literacy, machine-learning application, and the ethical issues around the use of AI.
Sceptics highlight the limitations of an AIdriven approach to business education. Despite its impressive features, ChatGPT is nothing more than a digital copy writer, and not even a reliable one. Oblivious to cheating, it has no qualms in producing content that is wrong, fictitious or unverifiable just to please its users. “Whilst machine-learning tools have proven to be adept at distilling large quantities of information, they cannot effectively assess the validity of these outputs in the wider context of the world we live in,” says Julia Handl who teaches Decision Science at Alliance Manchester Business School and leads an executive education programme on data and AI. For researchers, things are even trickier, given the gulf between natural speech and the scientific lexicon, according to Victor Botev, former AI research engineer and co-founder of Iris.AI, a startup using AI for scientific text understanding. What’s more, LLMs are terrible at finding and citing credible sources. “Although an enormous engineering challenge and likely not a priority for OpenAI’s developers, the bigger challenge will be to ensure that models are consistently selecting the right sources,” Botev says, adding: “They should be able to lay out to the end-user why any given source is the correct choice in factual terms that highlight its reasoning and driving factors in its decision-making.”
Overreliance on technology may also have other negative side-effects on business schools if they jump on the AI bandwagon too soon. “The potential impact of ChatGPT being misused by students could have serious consequences on the reputation of business schools that send into the job market graduates that bring less than expected value to a hiring organisation than commensurate with the standing of the school,” says Russell Miller, Director of Learning Solutions and Innovation in Executive Education at Imperial College Business School, adding: “The success of executive education is more often measured in terms of the business impact it creates and less about passing an assessment. Any misuse of ChatGPT is likely to be exposed quickly in the workplace.”
Testing the tests
One part of business education that faces imminent disruption is student assessment. Software able to write like a business analyst renders essay writing obsolete. Many business school academics are tweaking assignments and exam questions to make it harder for bots to find relevant material. Turnitin, a software programme used by universities to detect plagiarism, has developed a feature that identifies AI-generated fluff. However, some reject this strict approach, arguing that automating assessment will free up time for professors to have meaningful interactions with students. Tech enthusiasts even predict that in the future students may be asked to feed questions to AI software and assess its answers. In one case, Phanish Puranam, who teaches Strategy at INSEAD, asked students to submit ChatGPT responses to the assignment question and include in their papers an appendix explaining how they used it. Such a radical change may mark a shift to education models more focused on helping students identify the right questions, rather than providing answers, argues David Slocum, Professor of Practice at Thunderbird School of Global Management: “In the context of ChatGPT and AI, a process-focus can shift learning from being about answers and assessments to the ongoing practice of learning, development and leadership itself.”
Automation can make an academic’s life easier in other areas too. When Wharton’s Terwiesch turned to ChatGPT to come up with exam questions, he was surprised by the quality of the result; the bot was even able to add a humorous flavour, an improvement compared to his dry writing style, as he admits. Although he thinks that students should not be allowed to use these tools when taking exams, he believes there is a place for AI-driven assessment. “There are assignments for which we don’t care about the student’s answer, but the process of answering. The student emerges as a better version of herself, because she learns by applying the material,” he says, adding that AI tools could enable academics to push students further: “Because of ChatGPT we can create new ways for students to engage with the material, instead of just sitting in the library and writing an essay. We can create virtual experiences that can make learning more fun and invidualised.”
For optimists, generative AI tools bring us closer to the holy grail of business education: understanding what works and why. “Although we cannot measure power skills or human skills like decision-making with complete accuracy now, advances in AI will eventually help teachers to measure this better,” says Degreed’s Palmer, adding: “Not only will this be useful for feedback and assessment while an executive is learning with them, but it also provides more tangible evidence of an executive achieving a specific skill, which will support future job moves and career development.”
Back to the future
For the time being, the academic community oscillates between Luddism and unabashed tech-utopianism. Many academic journals have banned authors from using ChatGPT, while enthusiastic futurists predict that AI may prove as important as Gutenberg’s printing press. Some of its uses may be more prosaic, but still useful in a field like business education, subject to rapid internationalisation and specialisation. In the case of healthcare management for example, ChatGPT could help researchers and practitioners familiarise themselves with different healthcare practices, markets and cultures, says Radi Haloub, Deputy Director of Education at UCL Global Business School for Health.
As for those who have seen traditional business education survive previous technology trends, including the advent of the internet itself, everything must change for everything to remain the same. “The current education system will absorb these technological changes in the same way it has absorbed computers, email, Wikipedia, Google, YouTube or MOOCs,” says Dominik Lukes, Assistive Technology Officer at the University of Oxford and speaker for Bett, an edtech event organiser, adding: “They will change how people approach some aspects of learning and we will see adjustments in where they focus their learning efforts. But what it takes to learn – time, effort, relationships – is unlikely to change.”
