Stay up-to-date with free briefings on topics that matter to all Californians. Subscribe to CalMatters today for nonprofit news in your inbox.
Lea esta historia en Español
Democratic Assemblymember Jacqui Irwin, a former tech insider, is taking on the industry with a far-reaching bill that would require artificial intelligence developers to disclose what data they use to “train” their systems.
“Consumer confidence in AI systems has not grown at the same rapid pace as industry adoption,” Irwin said at a hearing last month. “Many consumers have valid questions about how these AI systems and services are created.”
The concern from Irwin about AI is notable since she may be the Legislature’s top expert on the tech industry and, occasionally, its champion.
As a young engineer at Johns Hopkins University’s applied physics lab, she was assigned to troubleshoot launches of the U.S. Navy’s Trident II nuclear missiles — making her the Legislature’s only actual rocket scientist.
She’s a former engineer for Teledyne Technologies, a global aerospace and tech conglomerate headquartered in Irwin’s hometown of Thousand Oaks.
She co-chairs the California Legislative Technology and Innovation Caucus and a national legislative task force on AI, cybersecurity and privacy.She also has been a favorite of Big Tech, once authoring legislation that critics accused of weakening California’s digital privacy protections on behalf of the tech industry to which she had close family ties. At the time, her husband was the chief operating officer of Amazon-owned Ring.The prominent lobbying group TechNet in 2017 declared her “Legislator of the Year.”
But now TechNet and nearly every other lobbying group representing major tech companies oppose her latest legislation, Assembly Bill 2013. The influential California Chamber of Commerce is also opposed to the bill, which the state Assembly voted 56-8 to move to the Senate last week.
Last month, the Chamber’s Ronak Daylami told the Assembly Privacy and Consumer Protection Committee, on which Irwin sits, that Irwin’s bill could expose tech firms’ carefully guarded trade secrets.
“While it may not be obvious on its face,” Daylami said, “the expertise and judgment, as well as the actual selection of data and datasets chosen to train a specific AI model, is itself proprietary.”
But Irwin said her bill would give consumers a powerful tool to better understand the emerging technology, which has raised privacy alarms after it was revealed that tech firms used facial recognition, social media posts and copyrighted material such as artwork and news articles to train their artificial intelligence software.
Irwin said the requirement to disclose training data could also help ward against potential biases in the AI software’s decision making.
She said the issue piqued her interest at a recent meeting of the National Conference of State Legislatures where she heard a doctors’ group discuss using AI in dispensing medication. The problem, she said, was that it wasn’t clear whether such systems had inherent biases since the companies aren’t required to disclose the data they used to train their systems.
She wondered: What if it was like a clinical drug trial that only tested the medication on white suburban men, instead of a diverse group of patients whose bodies might react differently?“With these AI medical devices, you really should know what is the group that it was trained on,” she said.
Putting it more broadly, Hayley Tsukayama, a legislative advocate for the Electronic Frontier Foundation, likened the disclosure requirements to being able to read a list of ingredients that go into a meal.
“The ingredients list is occasionally much easier to parse than trying to taste a dish at the end and trying to figure out what’s in it,” she said.
The AI disclosure bill is hardly Irwin’s first foray into regulating tech since she joined the Assembly in 2014.
Her office provided what it called a “non-exhaustive” list of 13 other tech and cybersecurity bills Irwin has authored, most of which passed. Some of them were also opposed by the tech industry, which has donated at least $288,000 to her campaigns over the years, according to the Digital Democracy database.
Since 2015, Irwin’s votes have aligned with TechNet’s position on bills 28% of the time, according to a Digital Democracy analysis.
Her most controversial tech legislation, though, was a 2019 bill that critics said would have weakened the state’s landmark California Consumer Privacy Act. The law gives Californians legal authority to order tech companies to tell them what personal information they have collected, and customers can tell the companies to delete it and not to sell it.
