In partnership with
This is a week more about the business end of AI adoption rather than another courtroom cautionary tale. The thread running through it is how firms actually choose, build, price and staff around these tools, and the lead is an unusually candid account from a City firm of how it picked its AI, with the data to show its working.
AI in Practice
How one City firm chose its AI, and what its method can teach yours
Charles Russell Speechlys (CRS) has published a report this month, produced with TLW Consulting, setting out how it ran a two-week, head-to-head comparison of five leading AI products. It put 46 lawyers, from trainee to partner, through the exercise, feeding the same non-confidential tasks into each product so it could, in its own phrase, compare apples with apples. The tasks mirrored the five things its lawyers already use AI for: answering legal queries, summarising documents and email threads, drafting clauses and statements, and reviewing contracts and reports. The exercise had a practical trigger, namely that the firm's custom-AI vendor, Springbok AI, was acquired, and it suddenly needed a replacement without running a string of separate pilots.
The finding worth sitting with is what actually separated the products. Speed and accuracy, the report says, have become baseline expectations rather than differentiators: what built trust was whether a tool showed genuine understanding of legal concepts, used terminology correctly, and grasped the complexity of the work. The firm scored legal competence above efficiency, which is close to the opposite of how most vendor marketing and most benchmarking is framed. Harvey came out as the clear winner across the tasks, though the more useful output for other firms is the method rather than the result.
Two details will resonate with anyone running a practice. The firm's disputes lawyers scored the products lower across the board, which suggests AI still struggles with the analytical complexity that contentious work demands. And drafting made up a surprisingly small share of use, around 7%, which the firm's head of legal technology, Tessa Bartley, puts down to its mature, automated precedent bank: where a firm already has gold-standard templates, generative drafting adds less than you might expect. Partners and trainees also wanted different things, with partners after issue-spotting and depth, and trainees after clear explanations and usable first drafts.
This author's take is that the report earns its place not by naming a winner but by showing that a firm can make an AI decision on structured evidence rather than a vendor demo and a hunch. Most firms will never run a 7,000-data-point exercise, but the underlying discipline is available to a practice of any size: define the tasks that reflect your actual work, run the same tasks across each tool, and score legal understanding rather than raw speed. This week's practice prompt is built to help you design a smaller version of it.
Takeaways
Act: Before your next AI purchase or renewal, write down the five or six tasks that reflect how your team actually works, and test any tool against those rather than against the vendor's demo script.
Watch: Whether other firms follow CRS in publishing their methodology and data, which would give the market something better than anecdote to benchmark against.
Risk: Choosing on speed and price alone. The firms that get this wrong will be the ones that bought the fastest tool rather than the one that best understood the law.
Read: Legal Futures, and the full report, Charles Russell Speechlys
On your radar
A change at the top of the judiciary's AI agenda: In her Mansion House speech on 1 July, the Lady Chief Justice, Baroness Carr, confirmed that the Master of the Rolls, Sir Geoffrey Vos, will retire from the Bench later this year, alongside the President of the King's Bench Division. Sir Geoffrey has been the judiciary's most prominent advocate for AI and the Digital Justice System, and his 'machine age' framing has shaped how the courts talk about these tools. This author wrote about recent speeches by Sir Geoffrey here. Why it matters for UK lawyers: the judicial approach to AI has been driven from the top by a small number of senior judges, so a change in personnel is a fair moment to ask whether that direction will shift. If your practice depends on the courts' digital reform timetable, keep an eye on who succeeds as Master of the Rolls and whether the Digital Justice System keeps its momentum. (Courts and Tribunals Judiciary)
Clifford Chance builds its own AI knowledge bank rather than buying one: Clifford Chance has launched an AI-enabled knowledge management platform, built with Microsoft and Epiq Advisory, giving its lawyers natural-language access to more than 400,000 internal documents that have been reclassified, summarised and permissioned, and reportedly delivered in about six months. Why it matters for UK lawyers: the headline is another large firm choosing to build rather than buy, but the transferable lesson for a smaller practice sits in the plumbing rather than the budget: the value came from cleaning up, tagging and permissioning the firm's own knowledge before any AI was pointed at it. Audit whether your own precedents and know-how are in a state an AI tool could safely use, before you spend on the tool. (Artificial Lawyer, Legal IT Insider)
AI starts to appear as a line on the client's bill: Legora (formerly Leya) has moved its top tier away from a flat per-seat licence to consumption-based pricing, where each AI 'run' is charged by the compute it uses and can be attributed to the matter that prompted it. The reasoning is that agentic tools burn very different amounts of compute from one matter to the next, so a fixed seat fee stops reflecting the real cost. Why it matters for UK lawyers: usage-based pricing makes the cost of AI legible client by client for the first time, which raises an immediate question about recovery and disclosure, namely whether AI is a disbursement, an overhead, or something you pass through. Check how your engagement letters and pricing treat AI cost before a client asks. (Legora, Legal Futures)
Entry-level hiring is flat, and AI is now the suspect: New data reported by Artificial Lawyer shows entry-level associate hiring in the United States has been flat for four years despite a growing legal market, and asks the obvious question of whether AI is quietly absorbing the work junior lawyers used to cut their teeth on. Why it matters for UK lawyers: the data is American and the causation is unproven, but the trainee and NQ model here rests on giving juniors exactly the research, summarising and first-draft work these tools now do fastest. The CRS study's finding that trainees and partners want different things from AI is the same problem seen from the other end. Ask how your junior lawyers are being trained to supervise AI output, not just to produce work faster with it. (Artificial Lawyer)
Ad Break
In order to help cover the running costs of this newsletter, please check out the advert below. In line with my promises from the start, adverts will always be declared.
We are grateful to Wispr for continuing to support this newsletter. Wispr Flow remains a frequently used tool by this author.
Speak naturally. Send without fixing.
Wispr Flow turns your voice into clean, professional text you can send the moment you stop talking. Not rough transcription you have to clean up. Actual polished text — ready for email, Slack, or any app.
Speak the way you think. Go on tangents. Change your mind mid-sentence. Flow strips the filler, fixes the grammar, and gives you text that reads like you spent five minutes writing it.
89% of messages sent with zero edits. Millions of professionals use Flow daily, including teams at OpenAI, Vercel, and Clay. Works on Mac, Windows, and iPhone.
For Review
Knowing what matters: a practical guide to choosing the right AI tools for your firm (Charles Russell Speechlys)
The full report behind this week's lead, with the methodology, the scoring approach and the underlying data set out in enough detail to adapt for your own firm. If you have any role in AI procurement, this is the version to read closely rather than the news write-up.
Read or listen: Charles Russell Speechlys
Law firms don't have an AI problem, they have a data problem (Artificial Lawyer)
A vendor byline, from the chief executive of Centari, but worth reading past the framing for an argument that ties this week's build-and-buy stories together: the constraint on getting value from AI is rarely the model, it is the state of the firm's own data and know-how. A useful companion to the Clifford Chance story if your firm is weighing up what to fix before it automates anything.
Read or listen: Artificial Lawyer
Practice Prompt
Try the below prompt to design a smaller, structured version of the head-to-head evaluation this week's lead describes, so you can choose an AI tool on evidence rather than on a demo. Ensure you fill in context and constraints and other aspects marked with {}. Remember to adhere to the Golden Rules and do not upload confidential or privileged information to public tools.
You are assisting a UK law firm that is choosing between AI tools. Your task is to design a structured, apples-to-apples evaluation the firm can run itself, as a planning aid only.
Context to apply:
- The firm: {size, practice areas, and the type of work that matters most, e.g. "10-partner commercial firm, heavy on property and disputes"}
- Tools under consideration: {e.g., "Harvey, CoCounsel, Legora" or "not yet decided"}
- How AI is actually used today: {the main tasks, e.g. "answering legal queries, summarising documents, first-draft clauses, contract review"}
- Who will test: {e.g., "two partners, three associates, two trainees"}
- Timeframe and constraints: {e.g., "two weeks, no confidential client data"}
Produce an evaluation plan under these headings:
1. Representative tasks
For each main way the firm uses AI, propose two or three concrete, non-confidential test tasks that reflect real work, and that can be run identically across every tool so the results are comparable.
2. Scoring rubric
Build a scoring sheet that weights legal competence (does the tool understand the concepts, use terminology correctly, grasp the complexity) above efficiency (speed, number of prompts). Make clear that speed and surface accuracy are a baseline, not a differentiator.
3. Coverage by practice area and seniority
Suggest how to capture whether any practice area is underserved (disputes work often is) and whether partners and junior lawyers rate the same tool differently, since they tend to want different things.
4. Data capture
Propose a simple, consistent way to record each tester's structured feedback per task, so the firm ends up with comparable data rather than anecdote.
5. Decision framework
Set out how to turn the scores into a defensible recommendation, including what would justify choosing a lower-scoring tool (for example price, data terms, or exit risk).
Constraints:
- {Add firm-specific constraints, for example "must keep data in the UK or EEA" or "cannot use client documents in testing".}
- Apply the law and regulatory framework of England and Wales throughout.
- Do not invent product features, pricing, or vendor terms. Where the position is not known, flag it as a question to put to the vendor.
- This is a procurement planning aid, not legal advice, and the firm remains responsible for its decision and for verifying any output the chosen tool produces.How did we do?
Hit reply and tell me what you would like covered in future issues or any feedback. We read every email!
Thanks for reading,
Serhan, UK Legal AI Brief
Disclaimer
Guidance and news only. Not legal advice. Always use AI tools safely.
Recommended Newsletters
Below are a few newsletters that I recommend, for various reasons. Check them out!
