BCAT: good predictor
The Legal Services Board (LSB) has warned the Bar’s regulator that it cannot keep changing the cut score for the test would-be barristers have to pass before being allowed to take the Bar professional training course (BPTC).
However, the LSB has approved changes to the Bar Standards Board’s rules that will allow it to lift the cut score this year after research showed that it was set too low to be effective.
The BSB reported in March that in its first year, just 13 students – 0.6% of the cohort – were unable to pass the Bar course aptitude test (BCAT) after one or more attempts. Further, the BCAT had made no difference to the profile of students on the course.
As a result it proposed lifting the cut score from 37 to 45, which in the first two years would have reduced the number of students progressing onto the BPTC who subsequently failed it by nearly 20%. The research also showed that of the BPTC students who scored less than 45 on the BCAT, only 3% went on to obtain pupillage.
“The data we have from the 2013/14 and 2014/15 cohorts on the BPTC is very consistent in indicating that a cut score of 45 would have significantly increased effectiveness of the BCAT in excluding students less likely to pass the BPTC from the course,” it told the LSB.
Previous changes to the rule mean that the LSB no longer has to approve any change in the cut score, but it did approve new wording to clarify that successful completion of the BCAT is a requirement for all individuals before they embark on the BPTC and that it was the responsibility of the BSB to set the pass score for the BCAT from time to time.
In its decision notice, the LSB said: “Whilst the cut score is no longer specified in the regulatory arrangements, we are grateful for the open and transparent way that the BSB has managed the proposed change to the score.
“It has helped us to understand the BSB’s evidence base… to support both its plan to change the cut score and the continued use of the BCAT in the selection process for the BPTC.”
But it continued: “Whilst such a change to the cut score would appear to be supported by the evidence at this time, we would be concerned if, having changed the pass mark, that too did not deliver the intended outcome of improving entry and exit standards to the BPTC, and there followed further changes.
“Multiple changes to the pass mark might be interpreted as the BSB not having the evidence to support its approach to rule-making.”
In its application, the BSB said: “This recalibration of the cut score is founded on a sufficiently substantial body of evidence to give assurance that no further alteration will be necessary in the foreseeable future.”
Further, it said that it was “clear that the combination of providers’ own selection process and the BSB’s requirements does not adequately ensure that the candidates making the investment in the BPTC are always those most likely to progress successfully”.
It said the raising the entry threshold to those who had obtained a 2:1 – as some BPTC providers seem to have done – would not do the job.
“There is a wealth of independent data to suggest that raising the minimum academic achievement required by the BSB e.g. to a 2.1 could potentially have a disproportionate impact on students from protected characteristic groups.”
The evidence also showed that degree classification of a First/2:1 was “not as good a predictor of performance on the BPTC as BCAT result”.
The new cut score will take effect from December. The BSB will carry out a five-year review of the BCAT in 2018.
Leave a Comment