As the California legislature reconvened in Sacramento in January with hopes for a more regular legislative session in 2021, it again returned its focus to address the potential for bias and discrimination from the use of automated decision systems (ADS) by businesses. Assemblymember Ed Chau, chair of the Assembly Privacy and Consumer Protection Committee, is spearheading a bill—AB 13, or the Automated Decision Systems Accountability Act of 2021. AB 13 would require any business in California that provides a person with a program or device that uses an ADS to “to take affirmative steps to ensure that there are processes in place to continually test for biases during the development and usage of the ADS.”

AB 13 would also require businesses to conduct impact assessments on their programs or devices to determine any disproportionate adverse impacts on protected classes, to examine if the ADS serves reasonable objectives, and to compare alternatives to ADS or reasonable modifications that may be taken to limit adverse consequences on protected classes. In addition, AB 13 would require California businesses to submit a report providing specified information about ADS impact assessments to the Department of Financial Protection and Innovation by March 1, 2023, and annually thereafter. Failure to provide the required annual report will lead to a civil penalty.

The inspiration for AB 13 comes from a 2019 report by the Brookings Institution’s Artificial Intelligence and Emerging Technology Initiative, which highlighted questions that the growing application of algorithmic or automated decision-making systems raises about the sufficiency of current laws to protect against discrimination in an increasingly algorithmic and automated world. AB 13 seeks to address these risks by highlighting the care and responsibility ADS developers and users must take when human decision-making no longer has the central role in how decisions are made.

Businesses that use an ADS may wish to heed the implications of AB 13, as this is the second year in a row that the California legislature has promulgated legislation addressing the potential deleterious effects from ADS usage. Businesses should evaluate how they utilize ADS in their operations and whether there is adequate oversight in the decisions and outcomes that result from the use of an ADS. While AB 13 has only been referred to the Privacy and Consumer Protection Committee and may never be enacted into law, businesses can still take preemptive steps now to ensure their automated decision systems are not overly biased or produce discriminatory outcomes.