Training Data for Machine Learning
What distinguishes artificial intelligence – or, to put it more academically, machine learning – is that not every outcome is the result of a completely fixed set of rules, but rather that the artificial intelligence has learnt what is most likely to be the correct result. This is particularly important when it is difficult or impossible to codify all possible scenarios into a set of rules – for example, when the problem is too complex, such as when a photo needs to be matched to a person’s face.
A large proportion of AI models learn from examples. In the case of facial recognition, this could involve, for instance, a large number of images of a person in which they can be recognised.
Supervised Learning
Face recognition is a classic example of what is known as supervised learning. This means that the training data consists of pairs: the input (the photo of the face) and the correct output (the name of the person pictured). Other examples of supervised learning include AI models for the sentiment (mood: positive or negative) of a text, where the training data consists of pairs (text + sentiment).
This type of learning is referred to as supervised learning because the correct answer – sometimes referred to as the ‘gold label’ – comes from a reliable source (such as a human supervisor). Figuratively speaking, the supervisor is like a teacher with an answer book (the problem plus the gold label or ‘solution’) who can assess the model’s attempts against it, thereby helping the model to make fewer mistakes.
Self-Supervised Learning
Self-supervised learning also requires training data and examples. In this case, however, the training algorithm derives the key information solely from the input provided. This learning method is often used, for example, in the currently very popular generative models (ChatGPT, Midjourney). In language models such as those behind ChatGPT, for example, part of a complete text is ‘blacked out’ (or ‘masked’) and the model attempts to guess what might lie behind the blacked-out (masked) section. Because it has already ‘seen’ a great many other texts, it can, so to speak, compile statistics. And here, it can check the correct answer itself by peeking behind the blacked-out section. Because in GPT-style models the last section of the text is masked, and they then essentially predict the continuation of a text, these are also referred to as generative or predictive models.
Quality and Quantity of Training Data
‘More is better’ was also a fitting mantra in machine learning. Even if some of the training data wasn’t ideal (e.g. not 100% correct, or with sub-optimal ground truth labels), on average it didn’t make much difference. Furthermore, increasingly large neural networks were being trained, which tend to simply ‘memorise’ the correct results if they do not receive enough input to recognise the features needed to make accurate predictions. This is referred to as ‘overfitting’.
However, a large amount of training data also means long training times. Particularly when running the training on expensive hardware, one might question whether the cost-benefit ratio is appropriate. There are also studies that provide recommendations regarding the relationship between the size of neural networks (specifically the number of trainable parameters) and the optimal amount of training data (e.g. https://arxiv.org/abs/2203.15556).
However, if our ‘the more, the better’ approach no longer applies without restriction, then we should probably also strive for better training data in order to save time and money whilst still obtaining well-trained models. This can be essential for success, particularly when it comes to fine-tuning (more on this later).
What are typical quality features?
First and foremost, the training data should accurately represent ‘reality’ – that is, the queries that the model will later have to process. If my customers all ask questions in German, a training dataset consisting solely of English text is, at the very least, not ideal. Furthermore, this means that the data should be diverse enough to cover the full range of future inputs. In addition, different types of queries should not be overrepresented. In sentiment classification, for example, the examples should not consist of just 10 positive texts and 1,000 negative ones.
To put it even more clearly: if I want to use an image recognition model to detect cancer, for instance, I should not have only images of healthy tissue or only images of cancerous tumours in the training dataset.
Test Data vs. Training Data
When reading or hearing about training data, one sometimes comes across the term ‘test data’, and less frequently ‘dev’ or ‘developer data’. Apart from their volume, these data sets differ from the training data set solely in their function. The idea behind this is that if you want to verify whether an AI model works well, you should test it using data other than that used for training. If we recall the concept of overfitting, for example, otherwise you would simply be checking whether the model has memorised the data well. The situation is similar with the development dataset – AI developers use this during the phase when not all the finer details (e.g. the so-called hyperparameters) of a model have been worked out and one is searching for “good settings” or even the best model type. However, the idea is the same as with the test data.
How you now divide your data into training, test and dev data also depends on the type of model you are training or its intended use. Often, the training data makes up the largest portion of the data, with the dev data being the smallest. Particularly with large language models such as GPT-4 and the like, the proportion of training data is orders of magnitude larger.
Test Data and Training Data Simply Explained Using an Example:
Imagine you’re a maths teacher and you know that last year’s maths test on the same topic is doing the rounds among the pupils and is being diligently learnt by heart. Now, of course, you could set the exact same maths test again, hoping that memorising the answers suits everyone better than actually understanding the problem. Hopefully, you’ll get a good exam result. But the teacher in you certainly wouldn’t be happy. They want to know how much the class has understood. So you probably won’t set exactly the same test again.
The same applies to training AI models. You have enough ‘tasks’, i.e. training data, up your sleeve so that you can use different data for learning (‘training’) and the class test (‘testing’). One set is the training data, the other the test data.
In AI, there are further subdivisions for the data (“task”) sets, because before you really train a model, you want to check whether you’ve chosen the right type of model, or whether you can improve certain fine-tuning parameters (this is referred to as “hyperparameter tuning”). After all, “real” training can sometimes take a long time or be expensive. For this purpose, it is common to use other data, known as “development datasets”, which are often not as extensive as the subsequent training and test data.
So, as a good ‘AI model trainer’, you have:
- development rates,
- training sets,
- test data sets.
Fine-Tuning
In an era where ever-larger models such as GPT-4, or open-source variants like Alpaca, are available – models so large that small, medium and even large companies lack the resources to train them – the concept of ‘fine-tuning’ is becoming increasingly popular. The aim is to specialise a very large model – one that already performs very well ‘out of the box’ on general tasks – to a greater extent. This then enables even better performance on the specific task. It is, of course, no surprise that fine-tuning is also carried out using examples. Due to the outstanding general capabilities of, for example, Large Language Models (LLMs), far less data is required here than was previously used to train the model itself. This is what makes fine-tuning so attractive, as it is often manageable even for companies, institutions or even private individuals on a smaller budget.
And What About the Chatbot?
There isn’t just ‘the one chatbot’. Here at moinAI, we use a variety of different models within a single chatbot, which work together to achieve outstanding quality. And even behind the flagship ChatGPT, there are various underlying concepts. The foundation is GPT-3.5 or GPT-4, which utilises self-supervised learning. However, it is known that so-called reinforcement learning was also employed here, where, to put it very simply, people provide direct feedback on whether the output was good or bad; this would not be described as training data.
In the simplest case, a chatbot is based on so-called intent recognition. This is a model based on supervised learning, which requires pairs of chat texts and the correct intent (in our terminology, the ‘topic’) to be identified. The number of examples required here also depends on the quality of the underlying models.
In this case, it is perhaps worth mentioning that the training really only involves recognising what topic the customer or chat partner is currently interested in. In such a case, the response has hopefully already been drafted in advance by an experienced editor; it is not part of the training data. The training data looks more like this:
1) Text: “Hello, there!” , topic: “Greetings”
2) Text: “Take care!” , topic: “Farewells”
However, if you do want to train a chatbot like ChatGPT that also gives ‘free-form’ responses, the training data tends to look something like this:
Input:
“Context”: “You are a chatbot for a company that offers Software as a Service, and you are trying to help customers get to grips with the product.”
“Previous chat history”: “Customer: Hello; chatbot: Hi, how can I help you?”
“Request”: “How much does a monthly subscription cost with you?”
Output:
“That depends on the licence you choose. The entry-level licence starts at 100 euros a month, and our premium licences cost 2,000 euros a month.”
Fortunately, models that work like ChatGPT have already been ‘pre-trained’ by others and understand quite a lot ‘out of the box’; in other words, you don’t start from scratch and therefore don’t need the trillions of training and test data points that have already made the model intelligent. See the section on fine-tuning. Imagine a dog that has already been house-trained by the breeder and perhaps understands simple commands like “Sit” or “Stop”. But you’ll still have to teach it how to bring you the newspaper to the sofa or pick up the children from school on its own.
Or simply admit to yourself that a dog isn’t suited to picking up children from school. This is roughly how it is, for example, with ChatGPT and its use in customer communication. It is a versatile tool that meets many requirements – but not the requirements placed on corporate customer communication.
Conclusion
We see: Even though there are more and more very good, “generally-educated” large language models (LLMs), the quantity and even more the quality of training data also determine the success of an “AI ecosystem.” A chatbot whose training data is combined with experience and industry knowledge (e.g. ecommerce, finance or publishers), achieves a significantly higher level of automation than a bot that has only been trained with generic, uncontrolled data.
As we can see, even though there are an increasing number of very good, ‘general-purpose’ large language models (LLMs), the volume and, even more so, the quality of training data also determine the success of an ‘AI ecosystem’. A chatbot whose training data has been selected based on experience and industry knowledge (e.g. e-commerce, finance or publishing) achieves a significantly higher level of automation than a bot that has been trained solely on generic, uncontrolled data.
