Law firms often evaluate AI tools by looking at the product name, the vendor, and the visible features. That is understandable. It is also increasingly incomplete.
The same product may offer several models, route requests through different providers, change its default model over time, or apply different data-handling rules depending on the plan, configuration, region, or integration being used. What appears to be one approved tool may actually represent several distinct governance decisions.
An AI tool should not be treated as fully approved until the firm understands the relevant model, use case, routing, data handling, retention, and configuration.
01 · The Interface Problem The product name does not tell the whole story.
A familiar interface can create a false sense of stability. Users may believe they are using the same approved capability even when the underlying model, provider, or data path has changed.
A vendor may replace one model with another, introduce automatic model routing, add a lower-cost model for certain requests, or give administrators the ability to choose between several providers. Those changes may affect accuracy, confidentiality, retention, jurisdiction, explainability, and suitability for legal work.
The governance decision therefore cannot stop at “We approved Product X.” It must also consider what Product X is doing at the time and under the settings the firm actually uses.
02 · The Decision Set What model governance actually includes
Model governance does not require a law firm to become an AI laboratory. It does require the firm to understand the practical choices that materially affect responsible use.
Model Selection
Which model or model family is approved for the intended legal use, and whether users may choose alternatives.
Data Retention
Whether prompts, files, outputs, or metadata are retained, for how long, and under whose control.
Training and Improvement
Whether firm information may be used to improve models, products, services, or related systems.
Model Routing
Whether the vendor automatically selects among models or providers based on task, cost, availability, or performance.
Subprocessors and Providers
Which outside entities may process firm information and whether those relationships change by feature or region.
Administrative Controls
Whether enterprise settings allow the firm to restrict models, disable features, control sharing, or enforce approved configurations.
These are not merely technical preferences. They shape whether a capability is appropriate for confidential information, client work, legal research, drafting, analysis, or other consequential tasks.
03 · Hidden Change Routing and model updates can alter risk without a new purchase.
Automatic routing can create multiple approval states
Some platforms select models dynamically. A request may be routed to a different model based on complexity, performance, latency, cost, or availability. That may improve the user experience while making governance more complicated.
If the firm evaluated one model but the product may silently use another, the original approval may not cover the actual system behavior.
Model upgrades may be material changes
Vendors frequently describe model changes as product improvements. Many are. But a model upgrade can also change data handling, output behavior, reasoning characteristics, available context, vendor dependencies, or the types of tasks the system can perform.
Enterprise controls can create a different product
The consumer and enterprise versions of the same tool may differ substantially. Retention, training use, encryption, access controls, logging, administrative restrictions, and contractual protections may all depend on the plan and settings.
Approving a brand name without approving the relevant plan, model, configuration, and use case creates a record that may look complete while leaving the real decision undocumented.
04 · Why It Matters Model choices can change legal and business exposure.
A law firm does not need to evaluate every benchmark or technical architecture. It does need enough information to decide whether the system is appropriate for the intended use and whether additional safeguards are necessary.
Poor visibility into model decisions can create several practical risks:
- Confidential information may be processed under terms the firm did not review.
- A model may be used for tasks beyond the scope of the original approval.
- Client questionnaire responses may not match the product's actual configuration.
- Users may switch models without understanding the governance consequences.
- A vendor update may materially change the system without triggering a new review.
- The firm's records may identify an approved tool but omit the settings that made it acceptable.
The answer is not to freeze technology choices. It is to create a governance process that recognizes when model or configuration changes are significant enough to require review.
05 · Practical Response How firms can govern model and configuration decisions
The most useful record is not simply a vendor name. It is a concise decision record showing what was approved, for which uses, under what settings, and with which safeguards.
Five practical steps
Identify the actual model and plan
Record the approved product tier, model or model family, and whether users may select alternatives.
Document material data-handling terms
Capture retention, training use, subprocessors, routing, access controls, and any settings that materially affect confidentiality.
Define permitted and prohibited uses
Connect the model decision to actual work: research, drafting, summarization, client data, privileged information, and human review.
Restrict configuration changes where possible
Use enterprise controls to prevent unreviewed models, features, connectors, or sharing settings from being enabled.
Review material vendor and model changes
Treat model replacements, new routing, altered retention, and major feature changes as governance events.
Conclusion The model behind the tool belongs in the governance record.
Law firms do not need perfect technical visibility into every AI system. They do need enough visibility to understand the material decisions that affect confidentiality, competence, client expectations, and responsible use.
As AI products become more dynamic, the model, routing, and configuration behind the interface will increasingly determine whether a tool is appropriate. Firms that document those decisions will be better positioned to adapt when vendors change, clients ask questions, or internal use expands.
The approved tool is only part of the decision. The approved model and configuration complete it.
