ECOA and Regulation B and AI use cases

Annex III point 5(b): AI systems intended to evaluate the creditworthiness of natural persons or establish their credit score are high risk, except systems used to detect financial fraud. Providers need risk management, data governance, logging and human oversight. Deployers must carry out a fundamental rights impact assessment before use (Article 27), and affected persons have a right to an explanation of individual decisions from the deployer (Article 86).

Annex III point 5(b): AI systems intended to evaluate the creditworthiness of natural persons or establish their credit score are high risk, and a real time BNPL underwriting model is squarely this use, since it is the system making the credit decision rather than a supporting tool. Providers need risk management, data governance, logging and human oversight; deployers must run a fundamental rights impact assessment (Article 27), and consumers have a right to an explanation of an individual decision (Article 86).

Annex III point 5(b) makes AI systems that evaluate the creditworthiness of natural persons or establish their credit score high risk. Scoring a company is outside that point, but a sole trader is a natural person, and a model that also assesses the personal credit of owners, partners or guarantors evaluates natural persons. The tier therefore depends on who the borrower is and whose creditworthiness the model assesses.

The drafting assistant does not assess creditworthiness, so on its own it is not the Annex III point 5(b) credit scoring system. It helps the lender meet the Article 86 right of affected people to a clear and meaningful explanation of decisions based on such a high risk system. If it is built into the scoring system it shares that system's high risk obligations; as a separate drafting tool its tier depends on its design and on how its output is reviewed. The follow up chat assistant must tell customers they are dealing with an AI system (Article 50).

Answering questions and giving indicative estimates from published rules is limited risk with an Article 50(1) disclosure that the customer is talking to an AI system. If the assistant evaluates an individual's creditworthiness to decide or filter access to a loan, it falls under Annex III point 5(b) and is high risk.

Classification and indexing of documents for a person to review may fall under the Article 6(3) derogation for narrow procedural tasks, if the provider documents that assessment. That derogation does not apply once the system profiles a natural person: calculating a named borrower's qualifying income from their pay stubs and bank statements evaluates that person's economic situation, which is profiling under GDPR Article 4(4). A system intended to calculate qualifying income for the credit decision is high risk under Annex III point 5(b), evaluating the creditworthiness of natural persons, whether or not a person reviews its output. Human oversight of that output is a separate obligation under Article 14, not a way to take the system out of the high risk category.

Recommending restructuring terms for individuals involves assessing their ability to pay, which can amount to evaluating the creditworthiness of natural persons under Annex III point 5(b). Human approval alone does not remove that: the Article 6(3) exception covers only systems that do not materially influence the decision, such as a narrow procedural or preparatory task, and never applies when the system profiles natural persons. A tool that only assembles the case file can fall under the exception; restructuring for companies is outside point 5(b).

An automated valuation model values the collateral, not the person, so it is not itself listed in Annex III; the EU Mortgage Credit Directive treats property valuation (Article 19) and the creditworthiness assessment of the borrower (Article 18) as separate steps, and Article 18(3) says the creditworthiness assessment must not be based predominantly on the value of the property exceeding the amount of credit, or on an assumption that the property's value will increase. The valuation becomes relevant to Annex III point 5(b), creditworthiness assessment of natural persons, only where its output is built into a separate system that evaluates the borrower's creditworthiness, and whether that happens depends on how the lender designs the credit decision, not on the valuation model itself.

Verifying documents and third party data for a person or system to use is a narrow procedural task under Article 6(3) when the credit decision itself is made separately. That derogation does not save the system if it profiles natural persons: Article 6(3)'s last subparagraph makes an Annex III system high risk regardless of the procedural task exception when it does. Calculating an applicant's income and checking it against what people in that job are normally paid, which is how Informed.IQ describes its Ally Financial deployment, evaluates a person's economic situation and is profiling under GDPR Article 4(4), so that step will usually be high risk unless it is strictly limited to checking document authenticity rather than calculating or assessing income. Annex III point 5(b) separately excludes AI used to detect financial fraud from the high risk credit scoring category, which covers this use case's fraud screening step on its own. The tier also changes if the verification result automatically decides or materially narrows a consumer's access to auto financing, since retail auto loan applicants are natural persons: that use falls under Annex III point 5(b) as evaluating creditworthiness directly.

Explaining products and capturing an application is limited risk with an Article 50 disclosure. If the assistant evaluates creditworthiness or filters applicants on its own assessment, it falls under Annex III point 5(b) and becomes high risk, so keep the decision in the governed credit process.

Drafting letters for human approval is not listed in Annex III. The decision the letter communicates may come from a separate high risk system, such as credit scoring (Annex III point 5(b)) or a public body's eligibility decision on benefits (point 5(a)); the drafting tool does not make that decision. Article 50(2) requires the provider of an AI system that generates text to mark the output as artificially generated, which puts this on the limited risk (transparency) tier; this includes an organization that builds its own drafting tool. Article 50(2) does not apply where the AI has only an assistive function for standard editing and does not substantially alter the input data or the semantics of the output.