Close is fine when you are curious. Close stops being fine the moment the answer goes in a client file with your name on it.
General-purpose AI has no way to know which federal rules changed last quarter, so the failures are consistent and quiet.
None of those announce themselves. The answer looks the same whether the underlying constant is current or two years stale, which is exactly why it gets pasted into a plan.
The ground moved. Any model trained before these changes will answer confidently using rules that no longer exist.
The borrower cannot tell. Neither can the advisor, unless they already knew the answer, in which case they did not need to ask.
Nobody sues over a rounding difference on a curiosity question. The exposure appears when a recommendation is made, documented, and relied on. At that point the standard is not whether the number sounded right. It is whether you can show where it came from.
Fluent, fast, and plausible. The same question can produce a different answer tomorrow. No citation, no version, nothing to point at in a file review.
The same inputs produce the same output every time. Tracked against the regulation as it changes. An unknown comes back as an unknown rather than a confident guess.
Indistinguishable on the screen. Completely different a year later.
Three things, none of which a chat window provides.
The same inputs produce the same output every time, rather than a fresh generation each time you ask.
Tracked against the regulation as it changes, not as it stood whenever the model was trained.
A missing input comes back as missing. Closer to a calculator that refuses to answer than a model that predicts text.
Import the client’s federal loan record instead of describing it in a prompt. Model the plans that borrower is actually eligible for under the rules in force today, and produce a report you can defend a year later.