Why ATOS are getting it wrong: Part 4 - Fallacies and starting positions

4.Base rate fallacy, prosecutor's fallacy, and neglecting the claimant's history

When a claimant arrives for an ATOS assessment, the assessor pretty much seems to view them as a blank slate with no history. So a person who has been disabled or ill and on incapacity benefit or ESA for many years starts from a position of no points he same way as someone who has only just claimed.

The first reason this is problematic is due to the way the criteria for ESA are now so stringent, that even someone who is clearly very disabled indeed only just scrapes through. This is especially likely to be the case if their functional problems are limited to just one area. The maximum that person can score is 15 points and the number of points they need is also 15. In other words there is no margin for error at all. But as with anything in the real world, there will inevitably be an error rate. As I discussed in an earlier post, if your error rate for a given assessment is 10% (which seems generously low) then this approaches 100% if you reassess the same claimant enough times.

This leads to the second problem. By its very nature, it is likely that most people on ESA will be entitled to it. This is because they need to, and have probably been advised to, claim it. They've also got through the initial claim stages, and have been signed off work by a doctor. They're therefore a self-selected sample. Although most people in the population are not entitled to ESA, the vast majority of them are also not claiming it. The confusion arises when assessors see claimants as no more likely to meet the criteria for ESA than any other member of the population, so start from a position that the person being assessed will likely fail. This is a form of the base rate fallacy, and biases the assessment. It's also a form of the prosecutor's fallacy.

Consider another example: you know that only 5% of teapots are valuable. 95% are worthless. You train as an assessor of teapots, and are asked to assess the teapots in someone's collection. Let's assume in this scenario that assessing teapots on appearance is possible, but notoriously difficult and you know you will often get it wrong.

What are the most important things you need to know about each of the teapots you assess?

• Its history - The process by which it got into that collection.
• What previous assessments were made of it that got it there?
• What evidence was used in those assessments?
• How reliable is that evidence?

The answers to these questions radically alters the likelihood that you should trust your own judgment that a given teapot appears at first glance to be valuable or not. And the odds that a given teapot in someone's collection will be worthless are most definitely not 95%.

Do ATOS assessors understand these issues? What do you think?

The solution

It's really quite easy. When a claimant is being re-assessed, they start with the number of points they were given at the previous assessment (including any appeal). Points can be added or removed, but the change must be specifically identified, and evidenced, in enough detail. Likewise, if someone was previously put in the support group, they start in it in the same way. This should have the effect of focusing the assessor's mind better onto what should be reasonably expected in that case.

This might not solve all the other problems but it should massively reduce the error rate caused by constant re-assessment. It should also satisfy IDS and Grayling's desire not to 'write people off', misguided as that is.

Why ATOS are getting it wrong - Part 3

3. Misinterpretation of descriptors

This is something Greg Wood has talked about a lot, so I won't repeat it all in so much detail. I highly recommend his posts on the subject - he is completely correct to assert that this is an area of significant concern. The descriptors for ESA are here: to get or retain ESA, the claimant must score 15 points. Points are combined, but only one descriptor from a single activity can apply at once.

Although the LIMA system seems to award or rule out some descriptors, it is also clear from the manual that the assessor has a certain level of input as to when to award a descriptor as well. Since whether a descriptor applies is a legal question and not a matter of personal discretion, it is vital that the correct legal test is applied.

One particular problem seems to be what I would call an improper 'all or nothing' test. This is where if the assessor rules out the highest descriptor but doesn't appear to bother to check whether a lower one might apply. For example, a claimant states they can manage to see family every day. No points are awarded for activity 16 - dealing with other people. But hang on: what about 16 (b) and 16 (c) - these could still apply, as they relate to unfamiliar people, so more information is needed before ruling them out.

Similarly, I've seen monthly blackouts scoring no points, despite apparently scoring 6 points for activity 10. The assessor claimed that monthly was not significant enough to score points - but that isn't right - it's not what the descriptors say. A personal view of what's 'significant' can't override the clear wording of the law, but that seems to be what is happening.


The converse of 'all or nothing' is the 'bottom up counting' error. This occurs when the findings suggest that a high level descriptor applies - maybe one scoring 15 points - but instead a lower descriptor is awarded that is also applicable. For example, if a claimant has two blackouts a week, both descriptors 10 (a) and 10 (b) apply. Since only one of them can be awarded, the higher of the two should be awarded. 'Bottom up counting' leads to the lower wrongly being awarded.


A person who should have scored 15 points from a single descriptor thus ends up with fewer, and is kicked off ESA. I've seen this happen to several people.

JSA sanctions and housing benefit

It seems to be a common misconception that if someone's JSA is sanctioned, then they also lose housing benefit. This was shown happening to someone on Eastenders, and the notorious @DietQueen bragged about this on Twitter really happening to someone.

Not only is this not correct (housing benefit cannot be sanctioned), but a JSA sanction actually has the effect of preserving entitlement to housing benefit under regulation 2(3):

(3) For the purposes of these Regulations, a person is on an income-based jobseeker’s allowance on any day in respect of which an income-based jobseeker’s allowance is payable to him and on any day—
(a)in respect of which he satisfies the conditions for entitlement to an income-based jobseeker’s allowance but where the allowance is not paid in accordance with section 19 or 20A of the Jobseekers Act(46) (circumstances in which a jobseeker’s allowance is not payable

A person who has lost JSA for other reasons may still receive housing benefit if they have little or no other income, but they would have to take certain steps to demonstrate the lack of other income (HB can be claimed on grounds of low income alone, without receiving any other benefits). But where a JSA sanction is in place, regulation 2(3) is very clear that JSA is treated as if it were still being paid, for HB purposes.

I had wondered whether councils were getting it wrong because they are not getting enough information about why JSA is being stopped. So I asked the DWP what information is being given.

It seems clear that enough information is being given to councils to enable them to see whether JSA has stopped because of a sanction or not. The question must therefore be asked: why are councils still suspending or even ending HB entitlement when they have no business doing so? And why are the DWP not setting them straight?

Why ATOS are getting it wrong - Part 2

2. LIMA, and abuse of the concept of the typical day



When ATOS carry out an assessment, the assessor uses a computer system called LIMA. The manual is available here: although it's an old version from the incapacity benefit days, it explains a great deal to anyone who has read an ESA report, so it obviously works in a similar way still.



The key thing to note is the way the 'typical day' box is used. The assessor asks the claimant what he or she does on a typical day, and the computer extracts keywords from this to suggest which descriptors might apply. The computer rules out descriptors that cannot apply due to the claimant's abilities on a typical day.



At this point, it is worth noting how the law views ability and inability. Since a case called Moyna, it is pretty well established that a broad view must be taken of fluctuating conditions, so that a person who can manage things only occasionally is not deemed able to do them often enough to count as able to do them at all. If a person is usually able to manage things, on the other hand, he or she is deemed able to do them.



Therefore the ATOS concept of the typical day is a potentially valid one, provided that activities entered here really do represent activities managed on a typical day. As you may have guessed, this is the problem.



I've seen an 'ability' to see a GP every 6 weeks entered in the 'typical day' box. This was then extrapolated by LIMA to deny the claimant points for ability to deal with unfamiliar people. There was also no finding that he managed even those 6-weekly occasions in a reasonable manner without getting too upset, contrary to further case law on what is meant by 'ability'.



Another common thing entered in the 'typical day' box is: turning up to the ATOS assessment! This is used to deny a range of descriptors, with incorrect findings (or no findings) as to the repeatability of having to do this on a daily basis, or how well the claimant coped with it on the one occasion, or whether excessive pain resulted. Other offending entries here are occasional hobbies, visits to family, cooking a proper meal once a week or fortnight, and going shopping once a week or fortnight.



No wonder that all of these things being done at once on a 'typical day' results in an adverse finding. Yet an ability to do those things occasionally is perfectly consistent with quite severe limitations.

Why ATOS assessments get it so badly wrong - Part 1

I've been back working on ESA appeals, representing appellants, for a few months again now.

In my previous post I looked at how ministers are very confused about what role ESA is supposed to have, and how this is causing some of the problems with it.

Now, in a series of posts, I'm going to look at the far more familiar problem of ATOS assessments being of very questionable quality in and of themselves, and will attempt to explain why I think this is the case.

Contrary to the popular view, I don't think that the explanation is that the government pays a bonus to find against claimants. Although this wouldn't necessarily surprise me, there are enough other problems that this additional factor isn't needed to explain what's going on. The present setup already makes sense without it.

Here's part one of why there are serious problems:-

1. Constant reassessment means errors are inevitable

When I was on ESA, I questioned the DWP's intention to reassess me after one year, as the condition I had was not ever likely to change and would always present the same barriers to work. The answer I received was that in effect I should be grateful, as most people are reassessed after three or six months.

In the previous post linked to above, I discussed why the government's explanation of why it thinks constant reassessment is good is flawed. ESA is supposed to support people with limitations towards work, so kicking them off it is clearly not conducive to providing that support.

The problem with constant reassessment is that it makes errors inevitable in the long term. If the error rate so that an ill claimant's ESA is wrongly stopped is 10%, then over the course of a  lifetime, a chronically ill person faces a probability approaching 100% of having their ESA stopped wrongly at some point. Having this happen even once is catastrophic for that person.

In practice the error rate is almost certainly higher than 10%.