How much can you trust an artificial intelligence with legal advice? AI is proving itself in more and more fields, and many people ask whether it can replace a lawyer. I ran a little experiment. I asked a popular AI chatbot to summarize the rules of temporary agency work, and then, as a lawyer, I reviewed its answer. The result is instructive. The artificial intelligence got a lot of things right, but it also made several serious mistakes that could have grave consequences in a real case. In this article I show what it got wrong, and why you shouldn’t yet trust it with your important legal matters.
I had long been planning to put artificial intelligence’s legal knowledge to the test. When it comes to AI, the question keeps coming up among lawyers too, whether these machines will take our jobs. One might as well start preparing for a battery-pod existence like in The Matrix, not to mention Skynet.
So the other day I asked a popular artificial intelligence chatbot to write me a short summary of temporary agency work. The task was not tricky, a purely basic question of labor law that any practicing lawyer could answer even if woken from a deep sleep. Let’s see how the machine performed.
Let me tell you in advance that the end result was mixed. There were parts that the artificial intelligence summarized surprisingly accurately, and there were parts where it was seriously wrong. That is precisely what makes it dangerous, because the correct and the mistaken statements sound equally confident.
Let’s start with the good news, because the AI gave the correct answer on several points.
It correctly established that the assigned worker is in an employment relationship with the temporary-work agency. The employment contract is concluded by the agency, and so only the agency is entitled to terminate it. This is one of the basic rules of temporary agency work, and the machine knew it well.
Let me briefly recall how temporary agency work looks in practice. The agency and the user company conclude a contract with each other, in which they agree on the terms and the fee. The agency concludes the employment contract with the worker. There is no contract between the user company and the worker, but the user company exercises some of the employer’s rights.
It was also correct to state that the agency is responsible for ensuring the worker has the appropriate qualifications. If I order a specialist to look after stingrays at the aquarium, and I get a bricklayer instead, then the agency has not performed properly, and the rays go hungry. In return, the agency is looking at a nice little damages lawsuit, because rays are not a cheap hobby.
And now comes the merciless critique, because the artificial intelligence made several serious mistakes.
According to the AI, appropriate working conditions and protective equipment should be provided by the agency. This is a factual error. These are in fact the responsibilities of the user company, so much so that the parties cannot even deviate from this by mutual agreement. Interestingly, the machine later, in the section on the employer’s rights, correctly assigned this to the user company, in other words it contradicted itself.
According to the AI, wages are paid by the user company. This too is a serious error. Paying the wages, the contributions, and the sick pay is the agency’s obligation. This is entirely logical, since the employment contract is created between the agency and the worker.
There was also a slip regarding fringe benefits. Providing these is a matter of the parties’ agreement, and the law does not clearly assign them to the user company. The benefits may also be paid by the agency.
There was also a turn of phrase stating that the agency is responsible for the ‘keeping’ of the employment contract. What exactly this means, the artificial intelligence did not reveal either. There simply is no such legal concept. Perhaps the machine was momentarily possessed by the spirit of some long-dead grammarian.
Overall, the artificial intelligence’s knowledge can be compared to that of a university student who woke up on Tuesday morning and realized that on Wednesday they were supposed to sit a labor law exam they hadn’t studied a word for. On Tuesday they dove headfirst into the statute book, and on Wednesday they produced something. There are vague memories, occasionally a glimmer of clarity, but overall a fairly grey fog hangs over the landscape. A lenient examiner might just give it a bare pass.
The problem is not even that the machine makes mistakes. The problem is that it makes mistakes confidently. It presents the correct and the false statements with the same calm assurance, so a layperson cannot tell which is true and which is not. With a temporary agency work contract, a single such error can cost serious money.
This is exactly why artificial intelligence today is still more of an enthusiastic intern than a reliable advisor. It can be a good starting point, it is great for orientation, but the final decision is not worth entrusting to it. The law is full of exceptions, cross-references, and details that depend on the specific situation, which only a practicing lawyer truly grasps.
Artificial intelligence is excellent for getting a broad orientation on a topic, or for understanding the basics of a legal concept. But when something real is at stake, signing a contract, a dispute, or a corporate decision, you need a human expert.
An experienced lawyer knows not only the rules but also the exceptions, the case law, and what is the right course of action in the given situation. Artificial intelligence cannot yet replace this, because it does not know your specific circumstances, and it takes no responsibility for its advice.
True, the machine is a quick learner, so it is not out of the question that one day it will be worth turning to it for legal advice. Perhaps one day it will pass the bar exam with flying colors. Until then, however, you are better off entrusting your important legal matters to a flesh-and-blood lawyer.
If you need reliable, human advice on temporary agency work, labor law, or any other legal matter, feel free to get in touch. Book a consultation, and you will get clear, responsible answers to your questions.