Rule-based vs AI adaptive learning

Share this post

Reading Time: 7 minutes

Nodes

Adaptive learning uses competence, behavioural and demographic data to tailor a digital learning experience around each learners unique needs. There’s a lot of hype around this area which might have you thinking its all about Artificial Intelligence (AI), but that’s not the case and there are two types of adaptive learning approaches: AI-based and Rule-based. Each will afford you different features, benefits and outcomes.

Understanding rule-based and AI-based adaptive learning

Rule-based adaptive learning allows digital learning experiences to be based on a predefined set of rules. The rules are defined as if/then conditions which enable a learning pathway to be built in real time when each condition is exercised. 

Simple rule-based approaches include test-out assessments (whereby a pre-course assessment will test understanding of learning objectives and then content the user is proficient in) and navigational branching (in which a user response to a question may trigger a specific sequence of content to be presented). Most content authoring tools and some Learning Management Systems (LMS) with built-in authoring tools support test-out and branching so these are very easy for Learning & Development practitioners to exploit.

Complex rule-based approaches can be achieved by using bespoke development to create highly personalised experiences in which large amounts of learning materials can be sequenced in real time depending on a sophisticated rules engine that can use competence, behavioural and demographic data to help decide which content is presented to the learner. 

AI-based adaptive learning is offered by a new breed of commercial, off the shelf products. These also adapt the learning experience in real time, but use AI models instead of predefined rules to determine the order of content delivery. Large numbers of micro learning resources, each aligned to specific learning objectives and competence levels, are sequenced automatically according to competence, behavioural and demographic data. The AI constantly learns from all of the data being generated by users of the system as well as data being fed into it from other systems, adjusting the content delivered to each learner accordingly. The more data it is fed, the more effectively the system can adapt the experience.

Advantages of AI-based adaptive learning 

It is most effective when used at scale. AI is most effective when it has lots of data and lots of content on which the machine learning models can be trained. So large courses with large audiences work best, and if these can be supplemented by data from other systems, then all the better.

It can be highly effective when you have large volumes of content. An AI based system can quickly make sense of large volumes of content, with the AI ensuring that the right learners get the right content at the right time. Some vendors utilise AI to help classify and tag the content too, however while this may accelerate course and content setup, it is not specifically powering the adaptive experience for learners.

It comes into its own on complex courses that run over long periods. This is why most take-up of adaptive learning to date is in the education sector where learning programmes may run over many months, and even a small learning unit such as a course module is likely to last some weeks. These courses have lots of content and generate lots of data that the AI can work with.

It requires little human intervention. Well sort of. The AI may do the heavy lifting around personalising the user experience, but it actually requires a lot of human intervention to get content into the right format and to get the system setup and trained. But once up and running, a course can potentially be delivered to tens of thousands of users with very little human intervention. 

Disadvantages of AI-based adaptive learning

It takes a long time to implement and train AI models. The early stages of a project can therefore be expensive and the full benefits not realised for some months. This raises questions about the quality of learning experience for the initial sets of users whose data are helping to train the AI models.

Using AI without human intervention can lead to unpredictable digital experiences. In the chatbot world for example, where AI has been delivering digital content for some years, it’s now accepted that human-in-the-loop approaches should be used as part of the overall experience design, taking over where the AI struggles.

Control over the sequencing of content is completely relinquished to the system. If we look again to the chatbot world for lessons in digital content delivery, despite all the over-hyped AI chatbot solutions, the most successful chatbot ever built is rule-based. Mitsuku, or Kuki as she is more affectionately known, has won the Loebner Prize Turing Test for 5 years running. Kuki’s developer Steve Worswick said in Sept 2020, “There’s no machine learning in Mitsuku/Kuki. It’s a rules based system so I can have complete control and explainability over all of its responses. Its conversational style is monitored and improved by me, a human. The only machine learning conversational bots I have seen cost a fortune to train and run, as well as requiring thousands of dollars in hardware just to talk to them.”

There is still a high human cost associated with course content creation and ongoing maintenance. Content quality is really important in educational contexts and even more so in highly regulated workplaces, so validation of content being input to the system is absolutely critical and requires human oversight. As course usage grows, the machine learning models also need regular refining and training. So from a learning design perspective there are few savings, and from a maintenance side there maybe even be a cost increase.

You generally can’t just plug an adaptive course into your existing learning platform. Most (but not all) adaptive vendors run courses from their own platform. To organisations wanting to go down this route, that can mean double the platforms, double the money.

Advantages of rule-based adaptive learning

Rule-based systems can be implemented faster and cheaper. Rule-based systems don’t require any data at all to start off with, just a set of decision trees. Once usage data is generated and behavioural patterns emerge, complex rule-based systems can be manually trained over time, which adds to the overall cost of ownership but does mean lower upfront costs than with AI.

The learning experience remains fairly structured and so less control over navigation is relinquished to the system. This means an ongoing narrative can be designed to take the learner through a course, something which is not really possible with AI based systems which instead sequence content together in ways the designer has no control over.

Works fine on smaller, shorter courses. Rule-based systems are not reliant on volume of content or data to be effective, hence an adaptive experience can be built into shorter courses with smaller audiences. 

It can be delivered via SCORM packages so it runs in your LMS. Yeah I know, SCORM is old and dying and all that. But like it or not, the reality on the ground in 2021 is that SCORM still stubbornly remains the dominant packaging standard for digital short courses. If your course can be tied to a single platform for its lifetime then great, you can forget all about SCORM. But if your course needs to run on multiple platforms, or if SCORM is the only packaging standard your LMS accepts (which is usually the case in corporate learning), then unfortunately SCORM compatibility remains crucial. If you have the time, and can be bothered, you could even look at SCORM sequencing for a lightly adaptive experience, although most people find it over-complicated and don’t bother. A few AI adaptive learning vendors do support SCORM by using an empty ‘wrapper’ package with content streamed into it from a cloud platform – technically this breaks the SCORM standard which dictates the package should be completely self-contained, but as this approach doesn’t affect portability between systems, most people are happy to overlook that.

You might be able to already create rule-based adaptive courses in your LMS. Some platforms, including the popular open source LMS Moodle and its corporate derivative Totara, have rule-authoring frameworks within their built-in course creation tools, that allow many different types of learning resources and activities to be sequenced in a myriad of different ways.

Disadvantages of rule-based adaptive learning

Continual analysis of the data and training the system is required, just as in an AI based approach. In complex rule-based systems there is still an ongoing training need so the associated training cost still needs to be factored into the budget. 

Low efficiency on large courses. The beauty of adaptive learning is that you’re not taking the sheep-dip approach to digital learning, so the content is tailored to each user. If someone demonstrates prior knowledge in a subject area then they should not have to take the entire course. It stands to reason that the more you can personalise the experience and reduce the time spent away from work, the lower the cost of training and the higher the employee productivity. So on a larger course with greater volumes of content, an AI based system will save more money and staff productivity than a simple rule-based system. A complex rule-based system would sit somewhere between the two.

Which one is right for you?

If you have an off the shelf authoring tool or an LMS that supports rule-based content authoring then this may be well suited if:

  • You don’t have a lot of data with which to adapt the learning experience
  • Your courses are short, probably just an hour or two
  • Your audience is small, maybe up to a few thousand
  • Money is tight for setup and build, and maybe non-existent for maintenance
  • You need a rapid course build and implementation
  • You want to retain a structured experience with a strong narrative
  • You want to package up as SCORM and run the programme in your corporate LMS.

You may want to consider developing a customised course with a complex rule-based approach when:

  • You do not have much data to start with but envisage generating lots over time
  • Your courses stretch from multiple hours to many days
  • Your audience is in the tens of thousands which will allow you to achieve the ROI for the development time
  • You can get budget to help train the model over time
  • You want some light structure to the experience but with freeform elements too
  • You want to package up as SCORM and run the programme in your corporate LMS

Consider using an AI-based tool when:

  • You have large amounts of existing data about learners or will generate lots of data as you go
  • You have a large volume of content and may also need help classifying and tagging it all
  • Your course is complex and may comprise weeks or even months of learning 
  • You envisage a high-scale audience, maybe hundreds of thousands of learners over time
  • You have a sizeable budget for setup and maintenance, and a strong ROI case to cover those costs once you scale
  • You have a 3 month window to implement a course and wait for it to become fully optimised
  • You are happy that the learning experience is freeform and unstructured with little in the way of narrative structure
  • You are ok with the implementing a new learning platform alongside your LMS.

Image credits: “Socialized Energy (AKA Smart Grid)” by labguest is licensed with CC BY-NC-SA 2.0. To view a copy of this license, visit https://creativecommons.org/licenses/by-nc-sa/2.0/


Share this post

4 thoughts on “Rule-based vs AI adaptive learning”

  1. I like and would agree with much of the above. One factor not mentioned is student behaviour. You can design as intricate an adaptive network as you wish but for example: if students are getting 15% of their university course credit by completing the adaptive component, they will, if they can, simply game the system to do so. That may manifest in students simply skipping the supplementary/remediation content and moving through the main content. That isn’t to say that adaptivity is pointless but I think there is a lot of work needed to refine courses, which can be time-consuming for hard-pressed academics who may see your course and the lengthy development cycle as just burdensome in the first place. One possible way forward is to put the content creation more in the hands of the instructor/teacher. I have seen this done and while it can create less adaptive courses it may just be that patience is required.

  2. The other aspect to CogBooks’ adaptive approach is the design of the content and the work done on establishing a ‘network of relations’, which is jargon for finding the relationships between different pieces of content. That might be deciding on whether something is a definition or whether it’s a fuller explanation or perhaps something of higher cognitive value. Once you have established those networks, it is possible to generate more than one course from the same overall network. It may also be the case that in establishing the network, you discover gaps in the content that you may wish to fill, depending on the class you are teaching. So, in essence, adaptive design is not solely about the technology; it’s about the design of the content too.

  3. Thanks for reading Andrew,  I know CogBooks have done a LOT of adaptive work in the higher ed space so it’s great to get your viewpoint. Really interesting point about the impact of adaptive design on student behaviour and attempts to game the system, one to look into further I think. I like your point about the time consuming nature of reworking and optimising material for adaptive delivery when instructors / course designers tend to be time pressed already. I guess it is in response to this that some of the vendors I’ve looked at have human-based content optimization services, some even have training courses for learning designers because it’s such a different concept from more traditional, ‘sequenced’ digital course. I don’t know whether these services are universal to all adaptive vendors, but clearly there is a need for human intervention. So I completely agree, there is a big impact on design here, the technology supports this but it is ultimately a human problem.

Leave a Comment

Your email address will not be published.