Right to explanation

From Wikipedia, the free encyclopedia

In the regulation of algorithms, particularly artificial intelligence and its subfield of machine learning, a right to explanation (or right to an explanation) is a right to be given an explanation for an output of the algorithm. Such rights primarily refer to individual rights to be given an explanation for decisions that significantly affect an individual, particularly legally or financially. For example, a person who applies for a loan and is denied may ask for an explanation, which could be "Credit bureau X reports that you declared bankruptcy last year; this is the main factor in considering you too likely to default, and thus we will not give you the loan you applied for."

Some such legal rights already exist, while the scope of a general "right to explanation" is a matter of ongoing debate.

Examples[]

Credit scoring in the United States[]

Further information: Credit score in the United States

Under the Equal Credit Opportunity Act (Regulation B of the Code of Federal Regulations), Title 12, Chapter X, Part 1002, §1002.9, creditors are required to notify applicants who are denied credit with specific reasons for the detail. As detailed in §1002.9(b)(2):[1]

(2) Statement of specific reasons. The statement of reasons for adverse action required by paragraph (a)(2)(i) of this section must be specific and indicate the principal reason(s) for the adverse action. Statements that the adverse action was based on the creditor's internal standards or policies or that the applicant, joint applicant, or similar party failed to achieve a qualifying score on the creditor's credit scoring system are insufficient.

The official interpretation of this section details what types of statements are acceptable. Creditors comply with this regulation by providing a list of reasons (generally at most 4, per interpretation of regulations), consisting of a numeric reason code (as identifier) and an associated explanation, identifying the main factors affecting a credit score.[2] An example might be:[3]

32: Balances on bankcard or revolving accounts too high compared to credit limits

European Union[]

The European Union General Data Protection Regulation (enacted 2016, taking effect 2018) extends the automated decision-making rights in the 1995 Data Protection Directive to provide a legally disputed form of a right to an explanation, stated as such in Recital 71: "[the data subject should have] the right ... to obtain an explanation of the decision reached". In full:

The data subject should have the right not to be subject to a decision, which may include a measure, evaluating personal aspects relating to him or her which is based solely on automated processing and which produces legal effects concerning him or her or similarly significantly affects him or her, such as automatic refusal of an online credit application or e-recruiting practices without any human intervention.

...

In any case, such processing should be subject to suitable safeguards, which should include specific information to the data subject and the right to obtain human intervention, to express his or her point of view, to obtain an explanation of the decision reached after such assessment and to challenge the decision.

However, the extent to which the regulations themselves provide a "right to explanation" is heavily debated.[4][5][6] There are two main strands of criticism. There are significant legal issues with the right as found in Article 22 — as recitals are not binding, and the right to an explanation is not mentioned in the binding articles of the text, having been removed during the legislative process.[5] In addition, there are significant restrictions on the types of automated decisions that are covered — which must be both "solely" based on automated processing, and have legal or similarly significant effects — which significantly limits the range of automated systems and decisions to which the right would apply.[5] In particular, the right is unlikely to apply in many of the cases of algorithmic controversy that have been picked up in the media.[7]

A second potential source of such a right has been pointed to in Article 15, the "right of access by the data subject". This restates a similar provision from the 1995 Data Protection Directive, allowing the data subject access to "meaningful information about the logic involved" in the same significant, solely automated decision-making, found in Article 22. Yet this too suffers from alleged challenges that relate to the timing of when this right can be drawn upon, as well as practical challenges that mean it may not be binding in many cases of public concern.[5]

France[]

In France the 2016 Loi pour une République numérique (Digital Republic Act or loi numérique) amends the country's administrative code to introduce a new provision for the explanation of decisions made by public sector bodies about individuals.[8] It notes that where there is "a decision taken on the basis of an algorithmic treatment", the rules that define that treatment and its “principal characteristics” must be communicated to the citizen upon request, where there is not an exclusion (e.g. for national security or defence). These should include the following:

  1. the degree and the mode of contribution of the algorithmic processing to the decision- making;
  2. the data processed and its source;
  3. the treatment parameters, and where appropriate, their weighting, applied to the situation of the person concerned;
  4. the operations carried out by the treatment.

Scholars have noted that this right, while limited to administrative decisions, goes beyond the GDPR right to explicitly apply to decision support rather than decisions "solely" based on automated processing, as well as provides a framework for explaining specific decisions.[8] Indeed, the GDPR automated decision-making rights in the European Union, one of the places a "right to an explanation" has been sought within, find their origins in French law in the late 1970s.[9]

Criticism[]

Some argue that a "right to explanation" is at best unnecessary, at worst harmful, and threatens to stifle innovation. Specific criticisms include: favoring human decisions over machine decisions; being redundant with existing laws; and focusing on process over outcome.[10]

More fundamentally, many algorithms used in machine learning are not easily explainable. For example, the output of a deep neural network depends on many layers of computations, connected in a complex way, and no one input or computation may be a dominant factor. The field of Explainable AI seeks to provide better explanations from existing algorithms, and algorithms that are more easily explainable, but it is a young and active field.[11][12]

Similarly, human decisions often cannot be easily explained: they may be based on intuition or a "gut feeling" that is hard to put into words. It can be argued that machines should not be required to meet a higher standard than humans.

See also[]

References[]

  1. ^ Consumer Financial Protection Bureau, §1002.9(b)(2)
  2. ^ US FICO credit risk score reason codes: Fundamental document from FICO listing all of the FICO credit score reasons that a score is not higher, March 31, 2010, by Greg Fisher
  3. ^ "ReasonCode.org | VantageScore Solutions". www.reasoncode.org.
  4. ^ Goodman, Bryce; Flaxman, Seth (2017). "European Union Regulations on Algorithmic Decision-Making and a 'Right to Explanation'". AI Magazine. 38 (3): 50–57. arXiv:1606.08813. doi:10.1609/aimag.v38i3.2741. S2CID 7373959.
  5. ^ a b c d Wachter, Sandra; Mittelstadt, Brent; Floridi, Luciano (December 28, 2016). "Why a Right to Explanation of Automated Decision-Making Does Not Exist in the General Data Protection Regulation". International Data Privacy Law. SSRN 2903469.
  6. ^ Is there a 'right to explanation' for machine learning in the GDPR?, Jun 1, 2017, Andrew Burt
  7. ^ Edwards, Lilian; Veale, Michael (2017). "Slave to the algorithm? Why a "right to an explanation" is probably not the remedy you are looking for". Duke Law and Technology Review. SSRN 2972855.
  8. ^ a b Edwards, Lilian; Veale, Michael (2018). "Enslaving the Algorithm: From a 'Right to an Explanation' to a 'Right to Better Decisions'?" (PDF). IEEE Security & Privacy. 16 (3): 46–54. doi:10.1109/MSP.2018.2701152. S2CID 4049746. SSRN 3052831.
  9. ^ Bygrave, L A (2001). "Minding the Machine: Article 15 of the EC Data Protection Directive and Automated Profiling" (PDF). Computer Law & Security Review. 17 (1). doi:10.1016/S0267-3649(01)00104-2.
  10. ^ EU's Right to Explanation: A Harmful Restriction on Artificial Intelligence, Nick Wallace, Center for Data Innovation, January 25, 2017
  11. ^ Miller, Tim (2017-06-22). "Explanation in Artificial Intelligence: Insights from the Social Sciences". arXiv:1706.07269 [cs.AI].
  12. ^ Mittelstadt, Brent; Russell, Chris; Wachter, Sandra (2019). "Explaining Explanations in AI" (PDF). Proceedings of the Conference on Fairness, Accountability, and Transparency - FAT* '19. New York, New York, USA: ACM Press: 279–288. doi:10.1145/3287560.3287574. ISBN 978-1-4503-6125-5. S2CID 53214940.

External links[]

Retrieved from ""