TO ETHICAL HORIZONS. THIS IS A
JOINT INFORMATIONAL HEARING OF
THE SENATE GOVERNMENT
ORGANIZATION COMMITTEE AND THE
SENATE BUDGET COMMITTEE, NUMBER
FOUR ON STATE ADMINISTRATION AND
GENERAL GOVERNMENT. TODAY WE'LL
BE DISCUSSING ARTIFICIAL
INTELLIGENCE AND HOW WE AS
POLICYMAKERS SHOULD PROCEED
WHILE NEW TECHNOLOGICAL
CAPABILITIES ARE QUICKLY
BECOMING AVAILABLE TO BE
INTEGRATED INTO AND UTILIZED BY
THE PUBLIC SECTOR. EVERYONE HERE
KNOWS THIS, BUT IT'S ALWAYS NICE
TO REPEAT IT. CALIFORNIA HAS
LONG BEEN AT THE FOREFRONT OF
GLOBAL ADVANCEMENTS IN EDUCATION
, RESEARCH AND INNOVATION. AND
NOW EACH ONE OF THOSE FIELDS IS
FUELING THE FIRE OF A NEW KIND
OF TECHNOLOGICAL REVOLUTION. A
REVOLUTION THAT IS SURE TO
IMPACT EVERY ASPECT OF OUR LIVES
, MUCH LIKE THE CREATION OF THE
PRINTING PRESS THE DAWNING OF
THE INDUSTRIAL REVOLUTION AND
THE INVENTION OF THE COMPUTER
AND LATER THE INTERNET ONCE DID
ARTIFICIAL INTELLIGENCE IS
EMERGING AS THE WORLD'S MOST
SIGNIFICANT TECHNOLOGY
ADVANCEMENT OF OUR TIME, WHILE
QUICKLY TRANSFORMING INDUSTRIES
AND INDIVIDUAL CALIFORNIANS
DAILY LIVES. WE SEE THIS EVERY
DAY AS OUR GREAT STATE HAS ONCE
AGAIN ESTABLISHED ITSELF AS A
WORLD LEADER IN A I AND
GENERATIVE ARTIFICIAL
INTELLIGENCE. CALIFORNIA IS HOME
TO 35 OF THE GLOBE'S TOP 50, A.
I FIRMS AND A QUARTER OF ALL A I
PATENTS AND CONFERENCE PAPERS.
HOWEVER THE GROWTH INTEGRATION
OF A I BRINGS SIGNIFICANT
CHALLENGE AND RISK TO EVERY ONE
OF US. JUST AS HUMANS HAVE
EXPLICIT AND IMPLICIT BIAS. A I
HAS THE CAPACITY TO ACT AS A
MIRROR BY REFLECTING AN
AMPLIFYING THOSE BIASES WHERE
TECHNOLOGY IS IMPLEMENTED
WITHOUT PROPER GUARD RAILS OR
SAFETY PRECAUTIONS. FOR THESE
REASONS AND MORE. LAST SEPTEMBER
, GOVERNOR NEWSOM ISSUED AN
EXECUTIVE ORDER ADDRESSING A I
IN ORDER TO PREPARE CALIFORNIA
FOR THE PROGRESS OF THESE
TECHNOLOGIES. THE GOVERNOR'S
EXECUTIVE ORDER ANNOUNCEMENT
REFER REFERENCE. EXCUSE ME. THE
ORDERS FOCUS ON SHAPING THE
FUTURE OF ETHICAL, TRANSPARENT
AND TRUSTWORTHY A I WHILE
REMAINING THE WORLD'S ARTIFICIAL
INTELLIGENCE LEADER. THIS
MORNING IS OUR OPPORTUNITY TO
DISCUSS ARTIFICIAL INTELLIGENCE
IN THE EXECUTIVE ORDER, WHICH
ADDRESSES SOME OF THE MOST
PRESSING ISSUES OF OUR TIME
OCCURRING RIGHT HERE IN OUR OWN
BACKYARD AND IN OUR DISTRICTS
ACROSS THE STATE. WE WILL BE
HEARING THE GLOBALLY RECOGNIZED
CALIFORNIA BASED ACADEMIC
EXPERTS IN A I AND TECHNOLOGY
POLICY. FROM A GLOBALLY
RECOGNIZED CALIFORNIA COMPANY
WITH LEADING A I TECHNOLOGIES
AND FROM OUR OWN LEADERS IN
CALIFORNIA STATE DEPARTMENTS
WORKING TO IMPLEMENT THE
GOVERNOR'S EXECUTIVE ORDER. I
LOOK FORWARD TO DISCUSSING SOME
OF THE MOST EXCITING
APPLICATIONS TO A I HAS TO OFFER
FOR THE STATE WHILE WE CONTINUE
TO APPROPRIATELY MANAGE AND
DISCUSS RISKS, AND I'D LIKE TO
INVITE OUR CO-CHAIR SENATOR
PADILLA TO MAKE HIS OPENING
REMARKS. THANK YOU. THANK YOU,
CHAIRMAN DODD. AND FIRST I WANT
TO THANK YOU FOR YOUR LEADERSHIP
AND FOR THE COLLABORATION AND
PUTTING THE JOINT INFORMATIONAL
HEARING TOGETHER. I THINK IT'S
GOING TO BE VERY INFORMATIVE AND
VALUABLE. I THINK WE ALL
RECOGNIZE THOSE THAT ARE
ENGINEERS TO THOSE OF US THAT
ARE MERE CONSUMERS AND LAYPEOPLE
ATTEMPTING TO TRY TO UNDERSTAND
AND MAKE GOOD PUBLIC POLICY THAT
THE EVOLUTION AND APPLICATION OF
THESE ALGORITHMS AND OF THESE
DEPLOYMENT UH, MODELS IS
REVOLUTIONARY, AND IT IS
SOMETHING THAT IS PROBABLY THE
MOST REVOLUTIONARY RECENT
DEVELOPMENT WITH INCREDIBLE
POTENTIAL TO IMPACT. THE GLOBAL
CONDITIONS SINCE THE DAWNING OF
THE DIGITAL AGE. FRANKLY IT HAS
INCREDIBLE POTENTIAL THAT HAS
CLEARLY BEEN DEMONSTRATED IN
IMPROVING EFFICIENCIES AND
OUTCOMES AND HAVING TRULY GLOBAL
REACH. IT IS INCREDIBLY UNIQUE
IN ITS NATURE IN THAT IT IS
ALREADY DEPLOYED. ACROSS MANY
SECTORS, THINGS THAT WE AREN'T
REALLY AWARE OF, OR THINK ABOUT
IN OUR DAILY LIVES. IT IS TRULY
CAPABLE OF REAL TIME DEPLOYMENT
AND APPLICATION ON A STATE
COMMUNITY, NATIONAL AND
INTERNATIONAL LEVEL, AND HENCE
THERE ARE CONVERSATIONS IN THE
EUROPEAN UNION AND OTHER PARTS
OF THE GLOBE. IN WASHINGTON WITH
THIS ADMINISTRATION. IN MANY
STATES. YOU DON'T NEED TO LOOK
VERY FAR TO SEE THE INCREDIBLE
NUMBER OF BILLS IN THE
CALIFORNIA LEGISLATURE THAT HAVE
BEEN INTRODUCED IN THE SESSION.
ATTEMPT TO BOTH UNDERSTAND AND
FIND AN EFFECTIVE PLACE FOR THE
STATE AND EFFECTIVE ROLE FOR THE
STATE IN THIS CONVERSATION. HAS
INCREDIBLE POTENTIAL. BUT IT
ALSO COMES WITH INCREDIBLE
RISKS. RISKS THAT REVOLVE AROUND
INEQUITIES ABOUT ACCESS,
DISPROPORTIONATE IMPACTS,
ADVERSARIAL AND OTHERWISE, TO
COMMUNITIES, LEAST ABLE TO BE
HEARD OR TO HAVE INFLUENCE IN
THE PROCESS OF DEVELOPMENT OF
GUARDRAILS STANDARDS OR
REGULATORY FRAMEWORKS. IT HAS
INCREDIBLE RISK WITH REGARD TO
THE DISCLOSURE OF SENSITIVE
INFORMATION, NATIONAL SECURITY,
PERSONAL PRIVACY, INTELLECTUAL
PROPHECY, UH, PROPERTY AND SO
ON. IT IS RAPIDLY GROWING AND
EVOLVING EVERY DAY. SO MUCH SO
THAT EVERY DAY THAT GOES BY
THOSE OF US IN THE WORLD OF
POLICY AND GOVERNMENT ARE
ALREADY BEHIND. THERE ARE GREAT
OPPORTUNITIES AND THERE ARE
GREAT VULNERABILITIES. AND AS
ONE OF THE LARGEST PURCHASERS ON
THE GLOBE OF TECHNOLOGY AND
APPLICATIONS AS STATE OF
CALIFORNIA HAS AN OBLIGATION AND
A ROLE TO CONTINUE TO BE A
LEADER IN THIS SPACE, MUCH AS SO
MANY OF OUR GREAT THINK TANKS,
EDUCATIONAL INSTITUTIONS. AND
REVOLUTIONARY TECHNOLOGY
DEVELOPMENT INTERESTS, MANY OF
WHOM ARE REPRESENTED HERE TODAY
HAVE ALWAYS BEEN ON THE
FOREFRONT AS CHAIRMAN DODD
MENTIONED OF THIS TECHNOLOGY.
FOR THE PURPOSE OF THE HEARING
TODAY IS TO MAKE SURE THAT WE
DON'T CONSTRAIN THAT INNOVATION
AND DEVELOPMENT. BUT ALSO THAT
WE HAVE AN INTELLIGENT AND
THOUGHTFUL UNDERSTANDING OF WHAT
THE GREAT RISKS ARE AND MAKE
SURE THAT WE CAN AVOID THOSE IN
A REASONABLE WAY WITHOUT
STIFLING INNOVATION. WITH THAT,
I LOOK FORWARD TO MUCH FORWARD
TO HEARING FROM OUR
DISTINGUISHED PANELISTS. THANK
YOU, CHAIRMAN DODD. THANK YOU.
CHAIRMAN PADILLA. AT THIS POINT
IN TIME, I'M GONNA ASK, UH,
MEMBERS OF THIS, UH, THEIR
SEATED AT THE DIOCESE IF THEY
WANT TO MAKE ANY COMMENTS,
RECOGNIZING THAT WE HAVE SOME
VERY IMPORTANT PEOPLE IN THE
AUDIENCE THAT ARE GONNA BE
PRESENTING TO US, AND THEY'VE
GOT TIME LIMITATIONS. UM I'M
GONNA START. UH UH, THE
JUDICIARY COMMITTEE CHAIR
SENATOR UMBERG ASKED TO BE
PARTICIPATE TODAY AND BOTH THE
CHAIRMAN AND I THOUGHT THAT WAS
APPROPRIATE. SO WOULD YOU LIKE
TO MAKE A FEW COMMENTS? THANK
YOU, SENATOR. THANK YOU, SENATOR
BADILLA FOR ALLOWING ME TO SIT
IN TODAY. UH, THIS IS AN
INCREDIBLY IMPORTANT ISSUE. IT'S
INCREDIBLY IMPORTANT ISSUE. BOTH
POLICY WISE AS WELL AS WITH
RESPECT TO THE LEGAL
IMPLICATIONS. THE TECHNOLOGY
SURROUNDING ARTIFICIAL
INTELLIGENCE IS A CHALLENGE.
IT'S A CHALLENGE FOR US POLICY
WISE WHEN THIS LEGISLATION THESE
BILLS COME TO JUDICIARY
COMMITTEE, WE HAVE JURISDICTION
OVER ARTIFICIAL INTELLIGENCE
LEGISLATION. FOR THE MOST PART,
UH, FORTUNATELY, I'M AN EXPERT
IN ARTIFICIAL INTELLIGENCE AND
FOR THOSE OF YOU WATCHING AT
HOME. MOST OF THE AUDIENCE IS
NOW SMILING. UH BUT THE LEGAL
IMPLICATIONS ARE HUGE. FOR
EXAMPLE, HOW DO WE RESOLVE
COPYRIGHT ISSUES? PATENT ISSUES
TRADEMARK INFRINGEMENT? HOW DO
THEY APPLY TO A I CREATIONS? IS
IT CLEAR WHO OWNS THE CONTENT?
WHEN GENERATIVE A. I CREATES
SOME CONTENT, UH, BEFORE WE
EMBRACE THE BENEFITS OF
GENERATIVE, A I LEGISLATURE AND
BUSINESS AND OTHER GOVERNMENT
ENTITIES. UH, NEED TO UNDERSTAND
THE RISKS AND HOW TO PROTECT
THEMSELVES AND HOW TO BE
TRANSPARENT SO THAT THAT WE HAVE
A FULL UNDERSTANDING OF THE
IMPLICATIONS OR AT LEAST AS FAR
AS WE CAN. TODAY OF ARTIFICIAL
INTELLIGENCE. WE'VE GOT A LOT OF
WORK BEFORE US, AND I'M GRATEFUL
AGAIN PADILLA AND SENATOR DODD
FOR EMBRACING THIS CHALLENGE,
GENTLEMEN. THANK YOU. THANK YOU
VERY MUCH, SENATOR ANYBODY ELSE?
SOUL. YEAH. UH, THANK YOU, MR
CHAIR. BOTH SHERIFFS, UH, AS THE
CHAIR OF THE MILITARY AND
VETERANS COMMITTEE. MY CONCERN,
OBVIOUSLY, IS THE SECURITY WHEN
IT COMES TO CALVET AND ALL ITS
INFORMATION IT HAS WITH ITS
VETERANS. UH, THEIR MENTAL,
PHYSICAL AND HISTORY AND ALSO
WITHIN THE A, UH, THAT, UH AND
OUR MILITARY AND IT'S SO
IMPORTANT THAT SECURITY IS JUST
THE TOP. 30 IN ALL A I THAT WE
DO, AND I'M SO HONORED TO BE
HERE. BUT MORE IMPORTANTLY, THAT
YOU'RE ALL HERE TO UNDERSTAND
HOW IMPORTANT THIS IS THE FUTURE
OF CALIFORNIA AND TO THE MEN AND
WOMEN WHO WORK EVERY DAY TO
DEFEND OUR COUNTRY AND
PARTICIPATE IN UNIFORM. WE'VE
GOT TO PROTECT THEIR PRIVACY. SO
WITH THAT, UH, THANK YOU, MR
CHAIR. THANK YOU. ANYBODY ELSE?
SEE, NONE WILL NOW MOVE TO OUR
FIRST PANEL. I LIKE TO INVITE
THE PARTICIPANTS TO THE
MICROPHONES AND ASK YOU TO
PLEASE INTRODUCE YOURSELVES AND
PROCEED WHEN READY. FIRST.
MICHAEL KAREN NICHOLAS. FROM U
èC L, A AND DANIEL HO. FROM
STANFORD. THANK YOU BOTH FOR
TAKING TIME OUT OF YOUR BUSY
SCHEDULES FOR COMING HERE TODAY
AND ADDRESSING THIS
INFORMATIONAL HEARING. SENATOR.
I GUESS THEY'RE GONNA YEAH, THAT
I THINK.
THANKS VERY MUCH FOR THIS
OPPORTUNITY. UM. MY NAME IS
MICHAEL KAREN COS I'M, THE
EXECUTIVE DIRECTOR OF THE U èC
L, A INSTITUTE OF TECHNOLOGY LAW
AND POLICY. AND AN AFFILIATED
FELLOW WITH YALE INFORMATION
SOCIETY PROJECT. UM INSTITUTE OF
TECHNOLOGY POLICY WAS
ESTABLISHED IN 2020 AS A
COLLABORATION BETWEEN THE U èC
L, A SCHOOL OF LAW AND THE SLY
SCHOOL OF ENGINEERING. AND AS
OUR INAUGURAL EXECUTIVE DIRECTOR
, I ESTABLISHED OUR MISSION TO
FOSTER RESEARCH AND ANALYSIS TO
ENSURE NEW TECHNOLOGIES ARE
DEVELOPED, IMPLEMENTED AND
REGULATED IN WAYS THAT ARE
SOCIALLY BENEFICIAL, EQUITABLE
AND ACCOUNTABLE, UM, ON THE LAW
SIDE OF CAMPUS, THAT MEANS
BRINGING, UH, A LOT OF PUBLIC
POLICY AND ENGINEERING EXPERTISE
TO FOCUS ON EMERGING TECHNOLOGY
POLICY CHALLENGES ON THE
ENGINEERING SIDE OF CAMPUS. UM,
THAT IS MOSTLY FOCUSED ON
DEVELOPING ETHICS CURRICULA FOR
COMPUTER SCIENCE ENGINEERING
STUDENTS, THOUGH WE ALSO DO SOME
, UM PURE COMPUTER SCIENCE WORK,
UH, RELATED TO A I
MISINFORMATION. I THE PROJECT
THAT I'M WORKING ON. THAT IS
MOST RELEVANT HERE IS WITH THE
ADMINISTRATIVE CONFERENCE ON THE
UNITED STATES, WHICH IS A YEAR
LONG PROJECT. UH, THAT WE KICKED
OFF IN OCTOBER TO DEVELOP
REGULATORY AND ADMINISTRATIVE
SAFEGUARDS AROUND THE USE OF A I
SYSTEMS ACROSS THE FEDERAL
BUREAUCRACY. UM AND I ALSO WANT
TO EMPHASIZE MY OWN BACKGROUND
IS IN LAW AND POLICY. ALTHOUGH I
TEACH, UH, ON THE ENGINEERING
SIDE OF CAMPUS, I SUPERVISE
RESEARCHERS FROM THE A GRADUATE
PROGRAM. I'M A CO ION PROJECTS
FOR EXPLORING MACHINE LEARNING
APPLICATIONS WITH ENGINEERING
AND COMPUTER SCIENCE
COLLABORATORS. BUT MY OWN
BACKGROUND IS NOT IN, UH, ON THE
TECHNICAL ENGINEERING SIDE,
THOUGH I DO HAVE ACCESS TO
RESOURCES ON THAT SIDE OF CAMPUS
, AND WE WILL BE DELIGHTED TO
FOLLOW UP. IF THERE ARE SPECIFIC
TECHNICAL QUESTIONS THAT WE CAN
BE HELPFUL WITH. WITH THAT SAID.
I WAS ALSO ASKED, UH, TOLD THAT
IT WOULD BE HELPFUL, UM, TO KICK
THIS OFF BY OFFERING A
DEFINITION OF A I THAT IS, UH
UH, CAN BE TRICKY, UM,
ESPECIALLY AS A RESULT OF THE
DELTA BETWEEN EXPERT AND LAY
UNDERSTANDINGS OF THE TERM. UM A
I SYSTEMS INCLUDE A NUMBER OF
DIFFERENT TECHNICAL CONCEPTS,
BUT ARE BROADLY UNIFIED THROUGH
THEIR AIM OF AUTOMATION, WHICH
APPROXIMATES HUMAN CAPACITY. UM
AND THEIR ABILITY TO LEARN FROM
EXPERIENCE. PROBABLY THE BEST
KNOWN TYPE OF SYSTEM WITHIN THIS
CATEGORY IS A MACHINE LEARNING
SYSTEM, WHICH IS THE FORM OF
ARTIFICIAL INTELLIGENCE
ALGORITHM THAT IMPROVES ITSELF
BASED ON TRAINING DATA. THE WAY
THE MACHINE LEARNS DEPENDS ON
THE ALGORITHMIC MAKEUP OF THE
SYSTEM. DEEP LEARNING AND ACTIVE
LEARNING ARE MORE ADVANCED
TECHNIQUES IN WHICH A SYSTEM
LEARNS HOW TO LEARN EITHER WITH
PREDETERMINED DATA SETS IN THE
CASE OF DEEP LEARNING OR WITHOUT
THOSE DATA SETS IN TERMS OF
ACTIVE LEARNING. MACHINE
LEARNING SYSTEMS CAN ESSENTIALLY
BE UNDERSTOOD AS ENORMOUS
STATISTICAL INFERENCE ENGINES
WITH THE CAPACITY TO GENERATE
OUTPUTS FROM THE ANALYSIS OF
LARGE INPUTS OF DATA. UM
IMPORTANTLY, THE DATA DEPENDENT
NATURE OF MACHINE LEARNING
TECHNOLOGY FORMS THE BASIS OF
BOTH THE POTENTIALS AND PITFALLS
OF CONTEMPORARY ARTIFICIAL
INTELLIGENCE. UM MANY CITE THE
TRANSFORMATIVE POTENTIAL OF
THESE TECHNOLOGIES, INCLUDING IN
THE IN THE PUBLIC SECTOR.
INNOVATION IN THE PROVISION OF
PUBLIC SERVICES IS VERY WELCOME
IN CALIFORNIA SHOULD EMBRACE THE
POTENTIAL OF THESE TECHNOLOGIES
AS A FORCE MULTIPLIER FOR AN
ADMINISTRATIVE STATE, WHERE
RESOURCES RESOURCE CONSTRAINTS
ARE A WAY OF LIFE. UM AND
THERE'S ALSO INTERESTING
POTENTIAL TO DEVELOP, UM,
EFFECTIVE RESPONSE STRUCTURES TO
CHALLENGES LIKE BIAS, SINCE
THESE SYSTEMS CAN BE
RECALIBRATED AND ALTERED MORE
EASILY AND EFFECTIVELY THAN A
HUMAN DECISION MAKER CAN UM,
HOWEVER, THE DATA DEPENDENT
NATURE OF THESE TECHNOLOGIES
MEANS THAT ALMOST BY DEFINITION,
THEY ARE BACKWARDS LOOKING AND
RESISTANT TO PROGRESS. MOREOVER
PUBLIC SUPPORT FOR THESE
TECHNOLOGIES DEPENDS NOT ONLY ON
THEIR PERFORMANCE BUT ON PUBLIC
CONFIDENCE OF THEIR EFFICACY AND
FAIRNESS AND THEIR ABILITY TO
MAINTAIN CERTAIN KEY ASPECTS OF
PROCEDURAL FAIRNESS. NSS A. I
RISK MANAGEMENT FRAMEWORK, BUT
IS QUITE SUCCINCTLY WHEN IT
STATES. WORTHINESS IS ONLY AS
STRONG AS ITS WEAKEST
CHARACTERISTICS. UM, DESPITE LOW
UNDERSTANDING OF A I SYSTEMS,
THEY ARE BEING ROLLED OUT TO
PERFORM A VARIETY OF TASKS
ACROSS DIFFERENT LEVELS OF
GOVERNMENT, INCLUDING BOTH IN
HOUSE PRODUCTS, AS WELL AS IT
WAS DEVELOPED BY THIRD PARTY
CONTRACTORS, EACH OF WHICH
PRESENTS NOVEL CHALLENGES TO
PRESERVING FAIR AND ACCOUNTABLE
GOVERNMENT AND MAINTAINING
PUBLIC LEGITIMACY, IN ADDITION
TO WELL DOCUMENTED CHALLENGES
WITH ACCURACY, BIAS AND DRIFT,
AS WELL AS MORE GENERAL CONCERNS
ABOUT A LOSS OF THE HUMAN
ELEMENT, UH, THERE ARE ALSO MORE
GENERAL CONCERNS ABOUT LOSING
THE HUMAN ELEMENT IN, UH
DECISION MAKING. I'LL, I'LL
CONCLUDE WITH A FEW THOUGHTS
ABOUT APPROPRIATE REGULATORY AND
GOVERNMENT RESPONSES TO THESE
CHALLENGES. UM TO START.
REGULATORY ATTENTION. TO PRIVATE
SECTOR COMPANIES THAT ARE
EXPERIMENTING IN THIS SPACE IS
WELCOME AND NECESSARY. HOWEVER I
WOULD NOTE, HOWEVER, I WOULD
SUGGEST THAT THE EXPECTATION ON
THE PUBLIC SECTOR UM WITH
REGARDS TO THINGS LIKE
TRANSPARENCY, RULES, ETHICS AND
SOCIAL RESPONSIBILITY RULES, ALL
OF THAT SHOULD BE HIGHER IN A
PUBLIC SECTOR CONTEXT THAN IT IS
IN A PRIVATE SECTOR CONTEXT DUE
TO THE HEIGHTENED IMPACT OF
PUBLIC SECTOR USES ON THINGS
LIKE HUMAN RIGHTS. UM I THINK
THAT INITIATIVES TOWARDS
TRANSPARENCY AND DISCLOSURE OF
THESE USES ARE WELCOME AND
NECESSARY AS A FIRST STEP
TOWARDS TRYING TO GET OUR ARMS
AROUND THIS ISSUE, BUT IT IS NOT
SUFFICIENT AS A SOLUTION. I
WOULD ALSO SUGGEST THAT THE
BREADTH OF THESE TECHNOLOGIES AS
WELL AS INCONSISTENCIES IN HOW
THEY ARE DEFINED AND USED
SUGGESTS THAT RATHER THAN
SEEKING TO BOIL THE OCEAN,
LEGISLATIVE AND REGULATORY
FRAMEWORKS SHOULD BE TARGETED IN
APPLICATION SPECIFIC IN ORDER TO
ENSURE THAT THEY ADOPT A POSTURE
, WHICH FITS THE POTENTIAL USE
CASES BEING CONSIDERED. I
WELCOME THE NOTION OF ITERATION
IN STANDARDS DEVELOPMENT,
ESPECIALLY GIVEN THE SPEED AT
WHICH THESE TECHNOLOGIES ARE
PROGRESSING. I WOULD URGE THE
SENATE TO CONSIDER TRADEOFFS
INHERENT IN SOME OF THE VALUES
THAT ARE BEING PURSUED, FOR
EXAMPLE BETWEEN INTERPRET AND
PRIVACY OR PREDICTIVE ACCURACY,
VERSUS INTERPRET OR PRIVACY
VERSUS ACCURACY, POTENTIALLY
IMPACTING FAIRNESS. UM THERE HAS
ALSO BEEN SIGNIFICANT EMPHASIS
ON TRADITIONAL PROCEDURAL DUE
PROCESS PROTECTIONS SUCH AS
TRANSPARENCY, REVIEW ABILITY,
SEPARATION OF FUNCTIONS,
PROVISION OF REASONS FOR A
DECISION. THERE. I WOULD SUGGEST
THAT. RATHER THAN SEEKING TO
PORT OVER THE SAME VALUES THAT
WE ACCEPT THAT WE EXPECT FROM
HUMAN DECISION MAKERS. THE
QUESTION SHOULD BE WHAT
UNDERLYING VALUES WE'RE AIMING
FOR. IN A ROBUST ADMINISTRATIVE
STATE AND A ROBUST SYSTEM OF DUE
PROCESS. AND WHETHER AUTOMATE
WHETHER AND HOW AUTOMATED
PROCESSES ARE CAPABLE OF FILLING
THESE SAME VALUES. UM THANKS
AGAIN FOR THE OPPORTUNITY TO
APPEAR, AND I VERY MUCH LOOK
FORWARD TO THIS DISCUSSION. CH.
UH CH P MEMBERS OF THE
COMMITTEES THANKS FOR THE
OPPORTUNITY TO TESTIFY TODAY.
UH, MY NAME IS DANIEL HO. I'M A
PROFESSOR OF LAW, POLITICAL
SCIENCE AND COMPUTER SCIENCE AT
STANFORD UNIVERSITY. I'M A
SENIOR FELLOW AT STANFORD'S
INSTITUTE FOR HUMAN CENTERED A.
I AND I SERVE ON THE NATIONAL A
I ADVISORY COMMITTEE AND I
DIRECT THE STANFORD REG LAB,
WHICH WORKS WITH A WIDE RANGE OF
GOVERNMENT AGENCIES AROUND A I
DEMONSTRATION PROJECTS.
CALIFORNIA IS THE INNOVATION
CAPITAL OF THE WORLD. IT'S THE
FIFTH LARGEST ECONOMY IN THE
WORLD. IT HOUSES 35 OF THE TOP
50 COMPANIES, AND IT'S
EDUCATIONAL INSTITUTIONS ARE THE
ENVY OF THE WORLD, FROM UC
BERKELEY TO STANFORD TO CALTECH
AND SINCE WORLD WAR TWO THE
NATION IN CALIFORNIA'S
PARTNERSHIP BETWEEN GOVERNMENT
UNIVERSITIES AND PRIVATE
INDUSTRY CATALYZED FUNDAMENTAL
ADVANCES THAT GAVE US MICROCHIPS
GP S¼ SYSTEMS AND THE WORLD WIDE
WEB, EACH OF WHICH SIT
ACCESSIBLY IN YOUR POCKETS
TODAY. YET THIS POST WAR MODEL
OF INNOVATION IS UNDER THREAT.
WHEN IT COMES TO A I FOR THREE
REASONS. ONE A. I RESEARCH HAS
BECOME SO CAPITAL INTENSIVE THAT
ONLY A HANDFUL OF PRIVATE
COMPANIES ARE AT THE FRONTIER OF
THE FIELD. VAST AMOUNTS OF
COMPUTATIONAL POWER AND DATA
REQUIRE HUGE CAPITAL INVESTMENTS
. THE INNOVATION ECOSYSTEM IS
INCREASINGLY BECOME MORE CLOSED,
CONCENTRATED AND OPAQUE. THAT
HURTS SCIENCE, ACCOUNTABILITY
AND INNOVATION. TWO. GOVERNANCE
OF THIS TECHNOLOGY HAS CALLED
FOR BY THE GOVERNOR'S EXECUTIVE
ORDER AND MUCH OF THE PROPOSED
LEGISLATION YOU'RE CONTEMPLATING
BEFORE YOU IS FUNDAMENTALLY
CHALLENGING, BECAUSE GOVERNMENT
LACKS A I EXPERTISE FEWER THAN
1% OF A PHDS PURSUE PUBLIC
SERVICE AND GOVERNMENT CANNOT
GOVERN A I IF IT DOES NOT
UNDERSTAND A I THREE. MANY
GOVERNMENTAL SYSTEMS STILL RELY
IN THE WORDS OF ONE OF YOUR
ASSEMBLY MEMBER COLLEAGUES ON
DINOSAUR TECHNOLOGY. SOME 46
MILLION AMERICANS TURNED TO
UNEMPLOYMENT INSURANCE DURING
THE PANDEMIC. BUT 12 STATES,
INCLUDING CALIFORNIA STILL RELY
ON SOFTWARE LANGUAGE, UH, DATING
TO 1959. THE SYSTEM AS A RESULT
BUCKLED GOING FROM A TIMELINESS
RATE OF 97% PRIOR TO THE
PANDEMIC TO JUST ABOVE 50% RIGHT
WHEN PEOPLE NEEDED UNEMPLOYMENT
INSURANCE, THE MOST SO MY CORE
MESSAGE TO YOU IN MY OPENING
REMARKS IS THIS A I PRESENTS AN
EXTRAORDINARY OPPORTUNITY FOR
THE GOVERNMENT AND THE PEOPLE OF
CALIFORNIA, BUT WE MUST REFRAIN
AND REJECT SCIENCE FICTION
IMAGES OF TECHNOLOGY REPLACING
HUMANS. THE BEST A I SYSTEMS
WILL SUPPORT HUMANS, NOT REPLACE
THEM. LET ME GIVE YOU ONE
EXAMPLE. AND DURING THE PANDEMIC
REGLA, WHICH I DIRECT WORKED
EXTENSIVELY WITH SANTA CLARA
COUNTY TO USE TECHNOLOGY FOR
COVID RESPONSE, FOR EXAMPLE, WE
DEVELOPED A SYSTEM TO HELP
BILINGUAL CONTACT TRACERS
CONNECT WITH PATIENTS THAT
NEEDED LANGUAGE ASSISTANCE. THAT
SIMPLE SYSTEM IMPROVED SAME DAY
CONTACT TRACING BY 12% HELPING
TO REACH THOSE HIT HARDEST EARLY
IN THE PANDEMIC. THEY DIDN'T
REPLACE BUT AUGMENTED THE
PANDEMIC'S FIRST RESPONDERS. AND
THIS KIND OF INNOVATION IS WHAT
WE NEED IN SPADES. AND COULD
MODERNIZE SYSTEMS LIKE THAT OF
UNEMPLOYMENT INSURANCE, HOWEVER,
REALIZING THESE BENEFITS
REQUIRES RESPONSIBLE APPLICATION
IN CALIFORNIA MUST LEAD THE
NATION BY EXAMPLE, IN
RESPONSIBLE GOVERNANCE WHEN
DEVELOPED POORLY. A I SYSTEMS
HAVE THE POTENTIAL TO WREAK
SIGNIFICANT HARMS FROM
AMPLIFYING BIASES TO SPEWING
MISINFORMATION TO ERODING
PRIVACY, JUST AS CALIFORNIA HAS
LED ON CONSUMER PRIVACY,
CALIFORNIA CAN HELP TO REBALANCE
OUR INNOVATION SYSTEM. LET ME
OFFER THREE RECOMMENDATIONS ON
HOW WE GET THERE. THREE
RECOMMENDATIONS ON PEOPLE,
INFRASTRUCTURE AND GOVERNANCE.
FIRST CALIFORNIA MUST NURTURE
DEVELOPMENT AND ATTRACT
TECHNICAL TALENT INTO PUBLIC
SERVICE. MOST IMPORTANTLY, THAT
MEANS UPSKILLING, NAMELY
PROVIDING OPPORTUNITIES FOR
CALIFORNIA'S GREAT CIVIL
SERVANTS TO LEARN ABOUT THE
POTENTIAL AND RISKS OF A I.
SIMILARLY CALIFORNIA SHOULD
FOLLOW FEDERAL MODELS OF HOW TO
BRING OTHER TALENT IN LIKE THE U
S¼ DIGITAL SERVICE, THE DIGITAL
SERVICE CORP AND PRESIDENTIAL
INNOVATION. IS TO HIRE A I
TALENT. CALIFORNIA'S GREAT
UNIVERSITIES OFFER AN
EXTRAORDINARY MECHANISM FOR
UPSKILLING CIVIL SERVANTS AND
BUILDING A TALENT PIPELINE INTO
GOVERNMENT. THE GOVERNORS JENNY
I. EXECUTIVE ORDER MANDATES THAT
AGENCIES CONSULT WITH UC
BERKELEY AND STANFORD AND
STANFORD. UH, HÌ A. I IS EXCITED
TO CO SPONSOR CALIFORNIA'S A I
SUMMIT, BUT CALIFORNIA SHOULD
ALSO FOLLOW THE LEAD OF THE
FEDERAL GOVERNMENT, WHERE
AGENCIES HAVE BUILT OUT ACADEMIC
PARTNERSHIPS TO BRING IN
EXPERTISE AND DEVELOP PATHWAYS
OF PUBLIC SERVICE FOR A I
STUDENTS. CALL FOR THE
CALIFORNIA A I RESEARCH HUB AND
A CALL FOR A STATE TALENT
EXCHANGE TO HELP UPSKILL AND
AUGMENT THE CIVIL SERVICE ARE
EXACTLY THE KINDS OF INITIATIVES
THAT CAN BE OF CENTRAL
IMPORTANCE IN HELPING THE
GOVERNMENT SEIZE THIS MOMENT.
CALIFORNIA HAS ONE, IF NOT THE
HIGHEST CONCENTRATIONS OF A I
TALENT IN THE WORLD AND IS IN
THE PERFECT POSITION TO LEAD THE
COUNTRY IN BUILDING NIMBLE
PROGRAMS TO STRENGTHEN THE
PUBLIC SECTOR AT THIS MOMENT.
SECOND. A MAJOR REASON WHY A I
HAS BECOME SO CONCENTRATED IN A
SMALL NUMBER OF TECH FIRMS LIES
IN TECHNOLOGY INFRASTRUCTURE.
THE FEDERAL GOVERNMENT HAS
RESPONDED BY PILOTING THE SO
CALLED NATIONAL A. I RESEARCH
RESOURCE OR THE NAIR TO
DEMOCRATIZE ACCESS TO COMPUTING
AND DATA RESOURCES BASED ON A
PROPOSAL AND WHITE PAPER FROM
STANFORD IN A FEDERAL TASK FORCE
, CALIFORNIA SHOULD LEAD THE
NATION IN THESE EFFORTS,
PROPOSALS FOR COMPUTING AND DATA
RESOURCES LIKE CAL COMPUTE AND
THE A. I RESEARCH HUB COULD
BROADEN ACCESS TO A WIDER RANGE
OF CALIFORNIANS THAT ARE
INCREASINGLY FINDING THEMSELVES
LEFT OUT. GOVERNMENT HAS ONE
MAJOR ADVANTAGE TO REALIGN
TECHNOLOGY WITH HUMAN VALUES THE
RELIABILITY OF GOVERNMENT DATA
CURRENTLY A I IS POWERED BY
HOOVERING UP ALL DATA ON THE
INTERNET AND LEARNING FROM IT.
GARBAGE IN GARBAGE OUT TOXICITY,
FALSEHOODS AND RISKS FROM GEN. A
ARE A RESULT OF THIS PARTICULAR
CHOICE BY PROVIDING SECURE
PRIVACY PROTECTED ACCESS TO MUCH
HIGHER QUALITY GOVERNMENT DATA.
THE NAIR AND CALIFORNIA
PROPOSALS CAN TURN A I TO SOLVE
MUCH MORE SOCIALLY USEFUL
PROBLEMS AND ALSO ENGAGE THE
FUTURE A I. TALENT OF THE WORLD,
WHICH IN THE WORDS OF ONE START
UP FOUNDER LIVE IN AN ECOSYSTEM
THAT RIGHT NOW HAS GOTTEN THE
WORLD'S BRIGHTEST MINDS AND GET
TO WORK ON THE SOCIALLY
IMPORTANT PROBLEM TO GET PEOPLE
TO CLICK ON ADS. ONE EXAMPLE OF
HOW WE REBALANCE THIS IS THAT
WHEN THE US GEOLOGICAL SERVICE
MADE SATELLITE IMAGERY AVAILABLE
FOR FREE, IT GENERATED THREE TO
$4 BILLION IN VALUE ANNUALLY AND
DROVE FORWARD OUR ABILITY TO
UNDERSTAND AND RESPOND TO
ENVIRONMENTAL THREATS. THE STATE
THAT PROVIDES THIS
INFRASTRUCTURE WILL CONTINUE TO
LEAD IN A I THIRD WHEN IT COMES
TO GOVERNING A I. CALIFORNIA
MUST ADDRESS THE HUGE
INFORMATION ASYMMETRY ABOUT NEW
AND HEIGHTENED RISKS POSED BY A
I THE MOST URGENT AND LOWER
BUDGET ITEM. REGULATORY SOLUTION
HENCE LIES IN A FORM OF ADVERSE
EVENT REPORTING, JUST AS IN
CYBER SECURITY, WHERE PARTIES
ARE REQUIRED TO REPORT
VULNERABILITIES AND ATTACKS. WE
NEED SUCH A SYSTEM FOR A I
ENABLING GOVERNMENT TO MONITOR
INVESTIGATE AND RESPOND TO
EMERGENT RISKS OF A I SYSTEMS.
IT'S A REAL PROBLEM WHEN ONLY A
SMALL NUMBER OF SELF INTEREST
SELF INTERESTED ACTORS HAVE THE
INFORMATION NECESSARY TO
DESCRIBE THESE RISKS. OVER THE
PAST YEAR, FOR INSTANCE, THERE
WAS A LOT OF WORRY ABOUT HOW GB
Tð LIKE SYSTEMS COULD HELP
HOSTILE ACTORS CREATE VIO
WEAPONS, BUT THE EVIDENCE WAS
PAPER THIN AND HAS BEEN
DEBUNKED. RAN FOR INSTANCE,
SHOWED THAT SUCH SYSTEMS A OFFER
NÓ INFORMATION BEYOND WHAT'S
READILY AVAILABLE ON THE
INTERNET AND B YIELD NÓ
DIFFERENCE IN THE ABILITY TO
CREATE A BIOATTACK PLAN.
GOVERNMENT MUST BE ABLE TO
SEPARATE HYPE FROM REALITY THE
WAY WE'VE ADDRESSED THIS IN
OTHER AREAS, CYBER SECURITY,
DRUGS, MEDICAL DEVICES,
PATHOGENS AND DESIGN DEFECTS IS
AN ADVERSE EVENT REPORTING AND
THAT'S WHAT'S NEEDED FOR A I
GOVERNMENT PROCUREMENT OF A I
TOOLS AND SYSTEMS CAN ALSO BE A
POWERFUL LEVER FOR INCREASING
TRANSPARENCY AND OTHER
TRUSTWORTHY PRACTICES.
CALIFORNIA CITIES ARE ACTUALLY
IN MANY WAYS LEADING THE WAY
LIKE SAN JOSE, WHICH IS, UH,
BEEN A LEADER IN BUILDING A
NATIONAL A. I COALITION TO
STANDARDIZE PROCUREMENT AND MAKE
SURE THAT VENDORS ARE SHARING
THE APPROPRIATE INFORMATION TO
GENERATE PUBLIC TRUST IN THE
GOVERNMENT'S USE OF A I SYSTEMS.
LET ME CONCLUDE WITH JUST TWO
LAST WORDS ON A I REGULATION,
WHICH I KNOW IS TOP OF MIND FOR
ALL OF YOU. ONE IS THAT WE MUST
BE EXTREMELY CAREFUL ABOUT
REGULATION THAT HAS THE
POTENTIAL TO ENTRENCH INCUMBENTS
AND QUASH OPEN INNOVATION. THE
POSTER CHILD OF THAT ILL
CONCEIVED THAT KIND OF ILL
CONCEIVED REGULATION IS A
LICENSING REGIME WHEREBY ONLY A
FEW WELL EQUIPPED COMPANIES
WOULD GET LICENSES TO DEVELOP
ADVANCED A. I MODELS. THIS IS
WRONG AND WOULD REDUCE OVERSIGHT
AND STIFLE INNOVATION. TWO IS
THAT MANY REGULATORY PROPOSALS
SINGLE OUT A. I SYSTEMS AS BEING
UNIQUELY RISKY AND SUBJECT TO
CONTROLS, BUT A I JUST AS IN THE
OPENING REMARKS BY SENATOR DODD
CAN BE A MIRROR AND ALSO EXPOSE
VULNERABILITIES OF EXISTING
HUMAN SYSTEMS IN DEVELOPING A I
SYSTEMS FOR THE IR S¼, FOR
INSTANCE, OUR TEAM UNCOVERED
DISTURBING RACIAL DISPARITIES IN
TAX AUDITS OF EXISTING LEGACY
SYSTEMS, NOT FANCY FORMS OF A I
CONCERNS ABOUT A I MAY ACTUALLY
POINT US TO OPPORTUNITIES FOR
THE REFORM AND IMPROVEMENT OF
EXISTING SYSTEMS IN SOME A. I
HAS TREMENDOUS PROMISE. BUT JUST
AS POST WAR CALIFORNIA SET US ON
THE PATH TO BECOME THE
INNOVATION ENGINE OF THE WORLD,
WE MUST TAKE THESE STEPS NOW TO
ENSURE THAT WE RETAIN THIS
LEADERSHIP BOTH FOR TECHNOLOGY
AND OUR HUMAN VALUES. I WELCOME
YOUR QUESTIONS. THANK YOU BOTH
VERY MUCH. BUT A COUPLE OF
QUESTIONS FIRST FOR DIRECTOR
CARMEN EOLAS AND THEN FOR
PROFESSOR HO. FIRST AND
FOREMOST, MY UNDERSTANDING IS
THAT BASICALLY THE FOUNDATION
MODELS THAT HAVE A BROAD,
BROADER APPLICATION AND KIND OF
THE GENERATIVE ENGINE. IF YOU
WILL, IN LAY TERMS, OR MAYBE THE
MODERN VERSION OF A I OPERATING
SYSTEM THAT ALLOWS THE
APPLICATION TO BE MULTITASK IN A
VERY BROAD SENSE. IS VERY DATA
INTENSIVE. A MASSIVE MIGHT BE A
BETTER A BETTER WORD. AND I'D
LIKE TO GET TO THE DIRECTOR. I'D
LIKE TO GET YOUR THOUGHTS ABOUT
SORT OF THE STATUS OF THE
CONVERSATION AROUND MISALIGNMENT
BECAUSE GOVERNMENT APPLICATIONS
OF THIS TECHNOLOGY FOR
GOVERNMENT AND PUBLIC PURPOSES
HAS A MORE SPECIFIC SCOPE THAN
THE GENERAL APPLICATIONS THAT
ARE DEVELOPED FOR THE FOUNDATION
MODELS, WHICH BASICALLY DRIVE
EVERYTHING, UH FROM THE PRIVATE,
INNOVATIVE SECTOR STANDPOINT.
AND WHAT IS THE CONVERSATION
AROUND? HOW WE ADDRESS THAT
MISALIGNMENT IN APPLICATION AND
HOW WE COULD APPROPRIATELY
SEGREGATE THOSE. AND THEN
SECONDARILY, I'D LIKE TO HEAR
YOUR COMMENTS ABOUT SPECIFICALLY
IN TERMS OF THE STATE'S
REGULATORY FRAMEWORK. HOW CAN WE
? HOW SHOULD WE BE THINKING
ABOUT MITIGATING OUTCOMES THAT
DO CONTAIN BIAS AGAINST
VULNERABLE POPULATIONS? THE MOST
VULNERABLE POPULATIONS WHO MORE
RELY ON GOVERNMENT? SOLUTIONS
ACCESS INFORMATION PROGRAM. AND
HOW WHAT ARE THE WHAT IS THE
THINKING AROUND THOSE TWO?
POINTS FIRST TO THE DIRECTOR. UM
. SO I'LL, UM. SO IN TERMS OF
THE MISALIGNMENT QUESTION. YEAH
CERTAINLY THESE APPLICATIONS ARE
ENORMOUSLY DATA INTENSIVE. THEY
ARE ALSO ENORMOUSLY ENERGY
INTENSIVE, WHICH IS
UNDERAPPRECIATED ASPECT OF THAT
CONVERSATION. UM, UH, I THINK
THAT YOU KNOW THE CORE QUESTION
COMES AROUND THE DEGREE TO WHICH
GOVERNMENTS ARE THE PUBLIC
SECTOR GENERALLY IS RELYING ON,
UH, A THIRD PARTIES TO DEVELOP
THE APPLICATIONS THAT THEY'RE
LOOKING FOR, AS OPPOSED TO
DEVELOPING THEM IN HOUSE. UM, I
THINK AS I MENTIONED IN IN, UM
I, AS I MAY HAVE MENTIONED IN MY
INTRODUCTORY, UM, COMMENTS EACH
OF THOSE PRESENTS, UM UNIQUE, UH
, CHALLENGES. UM YOU KNOW, UH,
DEVELOPING MORE SPECIFIC
APPLICATIONS INTERNALLY, UM,
ALLOWS FOR A MORE SPECIALIZED
CONTEXT, UM, AND ALLOWS FOR THE
PEOPLE THAT ARE DEVELOPING THE
NEW TECHNOLOGIES TO BE MORE
INTIMATELY FAMILIAR, UM, WITH
HOW THEY'RE GONNA BE USED AND
THE RISKS THAT WOULD POTENTIALLY
APPLY. UM, BUT OF COURSE IT'S
UH, CAN BE MORE AND LABOR
INTENSIVE TO DEVELOP THIS STUFF
IN HOUSE. UM AND YOU NEED TO
ATTRACT, UH, EXPERTISE, WHICH IS
WHICH IS COSTLY AND REQUIRES
LONGER TERM INVESTMENTS. UM UH,
ON YOUR SECOND QUESTION. UH,
REGARDING BIAS. UM I THINK ONE
OF THE TRICKIEST CHALLENGES
RELATES TO FACILITATING ROBUST
COMMUNITY INPUT INTO THE
DEVELOPMENT AND IMPLEMENTATION
OF THESE TECHNOLOGIES QUITE
OFTEN, WHAT WE'VE SEEN IN IN HOW
THEY'VE BEEN ROLLED OUT
PREVIOUSLY, UM, IS THAT THE
ORGANIZATIONS THAT ARE IN THE
SHARPEST EDGE OF THE
IMPLEMENTATION OF THESE TOOLS OR
SO SORRY THE PEOPLE IN
COMMUNITIES THAT ARE ON THE ON
THE SHARPEST EDGE OF
IMPLEMENTATION OF THESE TOOLS
CAN OFTEN SPOT THAT THEY'RE
RETURNING PROBLEMATIC RESULTS,
EVEN IF THE AUDITS ARE COMING
BACK CLEAN. OR EVEN IF, UM THE
FOLKS THAT ARE IMPLEMENTING THE
SYSTEM AREN'T SEEING AN ISSUE SO
ROBUST STRUCTURES OF COMMUNITY
ENGAGEMENT. I WAS JUST AS ONE
ASPECT OF THAT. UM UH, AS WELL
AS, UM ENSURING THAT THERE IS,
YOU KNOW. SUFFICIENT
OPPORTUNITIES FOR THESE
TECHNOLOGIES TO BE ASSESSED
AGAINST THE EFFICACY OF HUMAN
DECISION MAKERS OR THE
ALTERNATIVES AND WHERE THEY ARE
NOT RETURNING. THE DESIRED
RESULTS BE ALLOWED TO FAIL. UM,
YOU DON'T WANT TO END UP IN A
SITUATION WHERE, UM, PARTICULAR
INTERESTS ARE SO INVESTED IN THE
SUCCESS OF THESE TECHNOLOGIES
HAVE SUNK SO MUCH RESOURCES OR
OR OR, UH UH, CASH INTO THEM,
UH, THAT THEY'RE, UM THAT THAT
THEY DON'T ALLOW THEM TO, UM TO,
UH, BE ALLOWED TO FAIL. AND
THANK YOU, DIRECTOR AND TO
PROFESSOR HO. WE HAD A
CONVERSATION IN THE INNOVATION
SPACE. UH AND YOU TALKED ABOUT
THE LACK OF HOW DO WE ADDRESS?
HOW DO WE BETTER DEMOCRATIZE
THAT INNOVATIVE SPACE GIVEN THE
LIMITED NUMBER OF BROAD
PORTFOLIOS THAT HAVE A MAJORITY
OWNERSHIP STAKE IN THE
DEVELOPMENT OF THE OF THE
TECHNOLOGY TO DATE, WE CAN
PROBABLY POINT TO THREE OR FOUR
MAJOR PORTFOLIOS THAT
EXCLUSIVELY HOLD RIGHTS TO THIS
DATA, INFORMATION AND
TECHNOLOGICAL DEVELOPMENT. HOW
DO WE, YOU KNOW, INNOVATION
IMPLIES. A MORE MULTIFACETED,
OPEN AND INTEGRATED
CONVERSATION. UH, HOW DO WE
AVOID THAT MONOPOLY IN THAT
SPACE, AND SECONDLY, JUST
BRIEFLY YOUR THOUGHTS ABOUT HOW
THE STATE COULD APPROACH
PROCUREMENT? BECAUSE WE ARE A
VERY LARGE PURCHASER, AND I
THINK ONE OF THE THINGS THAT THE
STATE OF CALIFORNIA HAS GOING
FOR IT IS A MAJOR PURCHASER, AND
WE HAVE SOME SUBSTANTIAL MARKET
INFLUENCE. AND HOW DO WE, YOU
KNOW, LEVERAGE THAT IN A WAY
THAT'S POSITIVE. THANK YOU,
SENATOR. UH, TO TAKE THE FIRST
QUESTION FIRST. HOW DO WE DEAL
WITH THE KIND OF CONCENTRATION
UH, THAT EXISTS IN THIS SECTOR
BECAUSE THE RISE OF FOUNDATION
MODELS AS YOU HAD NOTED IN YOUR
FIRST QUESTION, UH, HAS MEANT
THAT THE FIXED COSTS FOR
TRAINING. THESE LEADING MODELS
IS EXTREMELY HIGH, REQUIRING
VAST AMOUNTS OF COMPUTING AND
VAST AMOUNTS OF DATA. I THINK
THERE ARE TWO THINGS REALLY, UH
UH, THAT ARE RELATED TO WHAT I
NOTED IN MY OPENING REMARKS THAT
I THINK A REALLY IMPORTANT
INTERVENTIONS HERE. ONE IS TO
MECHANISMS TO DEMOCRATIZE ACCESS
TO COMPUTE AND THIS, UH, KIND OF
DATA. SO THAT IS WHY THE CREATE
A I ACT, WHICH IS A BIPARTISAN
PROPOSAL AT THE FEDERAL LEVEL
FOR THE NA, THE NATIONAL A. I
RESEARCH RESOURCE REALLY HAS THE
POTENTIAL TO SIGNIFICANTLY
DEMOCRATIZE ACCESS, PARTICULARLY
WITH, UH, PUBLIC USE OR PUBLIC
SECTOR DATA IN MIND. AND THE
SECOND IS THAT I THINK WE SHOULD
BE REALLY CAREFUL WHEN WE THINK
ABOUT THE REGULATION OF A I
SYSTEMS THAT WE BE CLEAR THAT WE
WANT FORMS OF REGULATORY SYSTEMS
THAT HAVE SAFEGUARDS BUT ALSO
DON'T QUASH FORMS OF OPEN
INNOVATION BECAUSE THE HISTORY
OF CYBER SECURITY, FOR INSTANCE,
HAS BEEN ONE WHERE THE
TRANSPARENCY OF CYBER SECURITY
STANDARDS HAS ENABLED A LARGE
NUMBER OF INDIVIDUALS TO
IDENTIFY VULNERABILITIES OR, IN
THE WORDS OF, UH ONE A LEADER OF
THAT MOVEMENT, WITH MANY
EYEBALLS. ALL BUGS ARE SHALLOW.
UM, ON THE SECOND QUESTION. HOW
SHOULD THE STATE APPROACH THE
PROCUREMENT OF A I TECHNOLOGY? I
THINK THAT'S WHERE THERE IS
TREMENDOUS POTENTIAL TO GIVE YOU
A LITTLE BIT OF A SENSE OF THIS.
UH UH, THE FEDERAL GOVERNMENT
INCREASED ITS A I SPENDING BY
2.5 FOLD OVER THE COURSE OF FIVE
YEARS. IT IS UP TO THREE $0.3
BILLION NOW, UM AND, UH, I THINK
THERE ARE REALLY SIGNIFICANT IN
WHICH AS A PURCHASER, YOU CAN
IMPOSE STANDARDS ON WHAT VENDORS
ARE, UH WHAT SAFEGUARDS VENDORS
ARE REQUIRED TO TAKE NOW I THINK
THERE ARE ACTUALLY SOME REAL
CHALLENGES IN THE PROCUREMENT
SYSTEM OF HOW TO PROCURE A I,
MOST NOTABLY THAT PROCUREMENT
OFFICIALS HAVE TO ACTUALLY BE
TRAINED UP TO KNOW WHAT THE
RIGHT QUESTIONS ARE THAT THEY
SHOULD BE ASKING AND MANY OF OUR
PROCUREMENT SYSTEMS TREAT
SOFTWARE ACQUISITION OR SORT OF
LIKE HARDWARE ACQUISITION AND A
I IS FUNDAMENTALLY DIFFERENT
FROM PURCHASING A STAPLER. A. I
SYSTEMS ADAPT OVER TIME. UH, SO
IT'S NOT POSSIBLE TO EX ANTE
SPECIFY EVERYTHING THAT YOU NEED
OF A PARTICULAR SYSTEM AND WE
NEED MORE AGILE FORMS OF
PROCUREMENT SYSTEMS TO REALLY BE
ABLE TO ENSURE THAT THOSE
SAFEGUARDS ARE RIGHT THAT THERE
IS APPROPRIATE MONITORING, AND I
THINK SOME OF THE MOST
INTERESTING INNOVATION
INITIATIVES IN THE PROCUREMENT
SPACE HAVE ACTUALLY COME FROM
AGENCIES LIKE THE DEPARTMENT OF
HOMELAND SECURITY WHO HAVE STOOD
UP THINGS LIKE PROCUREMENT
INNOVATION LABS TO ADAPT
PROCUREMENT TO A I PRACTICES AND
LOCALITIES LIKE THE CITY OF SAN
JOSE, WHICH HAS BEEN UH UH A
MODEL IN TERMS OF THE
TRANSPARENCY OF HOW IT GOES
THROUGH THE PROCUREMENT PROCESS.
THANK YOU. THANK YOU VERY MUCH.
ARE THERE MEMBERS PARTICIPATING
WHO HAVE QUESTIONS FOR THIS
PANEL? SENATOR DODD. YEP. WHAT
ARE BOTH OF YOU WHAT IS NEEDED
TO BUILD A I LITERACY AMONG
PUBLIC SECTOR LEADERS AND
WORKERS TO ENSURE THEY ARE
EQUIPPED TO MAKE INFORMED
DECISIONS ABOUT A I PROCUREMENT.
FOR AND USE, FRANKLY. THANK YOU,
SENATOR DODD. UH, I THINK THERE
IS A LOT OF WORK THAT NEEDS TO
BE DONE. I THINK ON THE FEDERAL
LEVEL THERE WAS AN A I TRAINING
ACT THAT MANDATED THE DESIGN OF
AN ACTUAL SORT OF SERIES OF
COURSES, PARTICULARLY GEARED
TOWARDS PROCUREMENT OFFICIALS TO
GO BACK TO SENATOR PADILLA'S
QUESTION TO TRAIN PROCUREMENT
OFFICIALS IN HOW TO REALLY THINK
ABOUT THESE KINDS OF QUESTIONS.
I HAD THE FORTUNE OF TEACHING
ONE OF THESE MYSELF AND THE
GOVERNMENT ENDED UP OPENING THIS
UP TO THE ENTIRE CIVIL SERVICE.
AND THERE WERE 2000 ATTENDEES
BECAUSE THEY'RE SUCH EAGERNESS
BY THE CIVIL SERVICE. REALLY TO
LEARN ABOUT THIS, UH, EMERGING
TECHNOLOGY. UM SO I THINK WHAT
NEEDS TO HAPPEN IN THE SPACE IS
WE NEED TO FIGURE OUT TECHNICAL
TALENT PIPELINES INTO THE CIVIL
SERVICE THAT INCLUDES BOTH
UPSKILLING. IT INCLUDES THINKING
ABOUT DIFFERENT MODELS, SOME OF
THE VERY INTERESTING MODELS THAT
THE FEDERAL GOVERNMENT HAS TAKEN
ADVANTAGE OF IS THE USE OF
THINGS LIKE PRESIDENTIAL
INNOVATION, FELLOWS, THEY ARE A
LOT OF STANFORD. UH UH, SORT OF
GRADUATES, FOR INSTANCE, WHO
DON'T NECESSARILY WANT TO SPEND
THE ENTIRETY OF THEIR CAREERS IN
THE PUBLIC SERVICE, BUT MIGHT BE
WILLING TO THINK ABOUT A
ROTATION AND A SERVICE. WHERE
THEREFORE, A YEAR OR TWO YEARS
AND, AS A RESULT, TRANSFER A LOT
OF THEIR KNOWLEDGE TO THE CIVIL
SERVICE. IT'S ALSO WHY I'M SUCH
A BIG FAN OF THE PROPOSAL FOR
SOMETHING LIKE THE A. I RESEARCH
HUB THAT PROVIDES THAT COULD BE
COUPLED WITH A TALENT EXCHANGE
REALLY, TO DRAW ON THE EXPERTISE
OF CALIFORNIA'S GREAT
UNIVERSITIES TO BUILD OUT THAT
KIND OF TALENT. YEAH II, I WOULD
VERY MUCH AGREE WITH THAT. I
DEFINITELY, UM UH, SUPPORT A
GREATER EMPHASIS ON
INTERDISCIPLINARY ENGAGEMENT AND
BRINGING COMPUTER SCIENCE AND
ENGINEERING GRADUATES INTO INTO
THE PUBLIC SECTOR. I THINK THAT
THERE IS THERE IS ROBUST
PROGRAMMING TO TRY TO FOCUS ON
LAW AND PUBLIC POLICY, UM
GRADUATES IN THE SPACE AND
THERE'S LESS OF A PULL, UM
WITHIN STEM. I WOULD ALSO ADD
THAT MOVES TO EMPOWER THE
CALIFORNIA DEPARTMENT OF
TECHNOLOGY TO DEVELOP STANDARDS
IN THIS SPACE, UM ARE ARE A GOOD
START. UM UH, BUT WOULD UM,
SUGGEST THAT A SECTORAL SPECIFIC
APPROACH? BE A POSITIVE
DEVELOPMENT TO WORKING THERE. IF
I COULD SAY ONE MORE THING TO
SAY. IN RESPONSE TO THAT
QUESTION, THE NATIONAL SECURITY
COMMISSION ON A I HAD A REALLY
IMPORTANT NOTE IN ITS CHAPTER ON
HOW TO ACTUALLY GET TALENT,
TECHNICAL TALENT INTO GOVERNMENT
. AND OFTEN TIMES WE LAMENT THE
SORT OF PAY SCALE DIFFERENCES.
BUT WHAT THE NATIONAL SECURITY
COMMISSION ON A, I SAID, IS THAT
IT IS NOT JUST PAY SCALE
DIFFERENCES. IT'S ALSO THE
PERCEPTION AND OFTEN THE REALITY
THAT IT'S REALLY HARD TO DO
MEANINGFUL TECHNICAL WORK WITHIN
GOVERNMENT. I'LL GIVE YOU ONE
EXAMPLE FROM A P HÌ Dð STUDENT
WHO WANTED TO DEVOTE HIS CAREER
TO PUBLIC SECTOR TECHNOLOGY. A
MACHINE LEARNING P HÌ Dð STUDENT
FROM STANFORD, UH, WE HAD HIM
INVOLVED IN A COLLABORATION WITH
THE FEDERAL AGENCY, AND IT TOOK
A YEAR SIMPLY TO GET THAT PERSON
ACCESS TO THE REQUISITE
COMPUTING TO BE ABLE TO RUN
ANYTHING THAT WOULD BE SORT OF
MODERN FORMS OF MACHINE LEARNING
. AND AS A RESULT, THAT STUDENT
GOT SO FRUSTRATED AND ENDED UP
GOING TO THE PRIVATE SECTOR.
THAT'S THOSE ARE THE KINDS OF
THINGS THAT I THINK WE NEED TO
LEVEL AND I'LL ADD ONE OTHER. UH
UH, SORT OF NOTE HERE, WHICH IS
THAT WE HAVE A LOT OF FEDERAL
FUNDING FOR A I RESEARCH
INSTITUTES. WE HAVE A I RESEARCH
INSTITUTES FOR AGRICULTURE FOR
WEATHER FOR EDUCATION. WE NEED
ONE FOR GOVERNMENT SERVICES AS
WELL. SO JUST A QUICK FOLLOW UP.
YOU'VE YOU'VE TALKED ABOUT?
REALLY? YOU KNOW THE PUBLIC
SECTOR MEETING. YOU KNOW THE
PEOPLE IN OUR DEPARTMENTS AND
ALL THROUGHOUT BUT YOU'RE
LOOKING AT A WHOLE BUNCH OF
POLICYMAKERS. WITH ALL DUE
RESPECT TO MY COLLEAGUES,
GENERALLY, WE ARE JACK OF ALL
TRADES AND MASTERS OF NONE. SO
WHAT KIND OF RECOMMENDATIONS DO
YOU HAVE FOR US TO BE? COME
BETTER STEWARDS OF THIS
PARTICULAR POLICY AREA. UH TO,
UH, MAKE BETTER DECISIONS, UH,
ON BEHALF OF THE STATE OF
CALIFORNIA AND THE PEOPLE WE
SERVE I MEAN, I DON'T I DON'T
KNOW THAT YOU NECESSARILY NEED
TO HAVE YOUR OWN EXPERTISE IN
THIS SPACE. YOU JUST NEED TO
HAVE PEOPLE THAT YOU CAN RELY ON
TO HAVE THAT EXPERTISE. UM SO I
THINK THAT, UH, BUILDING UP, UM,
BUILDING UP SPECIALIZED AGENCIES
AND PROVIDING THAT PIPELINE?
YEAH II. I VERY MUCH AGREE, UM,
ESPECIALLY BECAUSE JUST
AWARENESS OF OPPORTUNITIES
WITHIN THE PUBLIC SECTOR AND
WITHIN GOVERNMENT IS JUST SO
MUCH LOWER. UM, IN TERMS OF THE
SCHOOL OF ENGINEERING, UM YOU
KNOW, YOU TALK TO ANYBODY AT THE
LAW SCHOOL. HALF OF THEM WILL
TELL YOU THAT THEY'RE YOU KNOW
THAT THEY'RE INTERESTED IN
PUBLIC. IN PARTICULAR WAYS, OR
THEY'RE INTERESTED IN GOVERNMENT
OR RUNNING FOR OFFICE. AND
THERE'S JUST THERE'S A HUGE
AWARENESS OF THAT SPACE. AND YOU
KNOW, I TEACH FIRST YEAR
ENGINEERING STUDENTS AND THE
LEVEL OF AWARENESS IS MUCH, MUCH
LOWER ABOUT OPPORTUNITIES, AND
THERE'S YOU KNOW, THERE'S A
RANGE OF REASONS FOR THAT, BUT I
THINK THERE'S A HUGE AMOUNT THAT
CAN BE DONE IN ORDER TO DEVELOP
PROGRAMMING TO GET THE WORD OUT
THAT THESE OPPORTUNITIES EXIST
AND HOW TO PURSUE THEM. THANK
YOU. THANK YOU. SENATOR NOTES
DIDN'T MEAN TO INTERRUPT YOU.
PLEASE PROCEED. I'M SORRY. I
JUST, UH I THINK TO CONNECT THE
EARLIER QUESTION BY SENATOR
PADILLA TO YOUR QUESTION,
SENATOR DODD. YOU KNOW IT IS A
REAL CHALLENGE, GIVEN HOW
RAPIDLY THE TECHNOLOGY IS
EMERGING. SENATOR LÀ HAD ASKED
EARLIER ABOUT THE EMERGENCE OF
THE FOUNDATION MODELS. THE EU A
I ACT WAS NEGOTIATED. BEFORE THE
RISE OF FOUNDATION MODELS, AND
THE ENTIRE RISK BASED FRAMEWORK
REALLY SAT VERY UNEASILY WITH
THE RISE OF FOUNDATION MODELS.
AND SO THERE WAS A SCRAMBLE
TOWARDS THE TAIL END TO
BASICALLY, YOU KNOW, STAPLE SOME
OTHER PROVISIONS ON THERE THAT
ARE SPECIFIC TO FOUNDATION
MODELS. SO I DO THINK THAT HOW
TO ACTUALLY LEGISLATE AND
REGULATE IN WAYS THAT ARE AGILE
AND DON'T REGULATE TO
YESTERDAY'S TECHNOLOGY WILL BE
REALLY IMPORTANT. I DO THINK
THAT WHAT WE DO ACROSS
GOVERNMENT RIGHT IS WE HAVE
AGENCIES WHERE WE BRING IN
EXPERTISE. TO REALLY HELP TO
GOVERN IN THE SPACE. I'LL TELL
THE TELL YOU ONE STORY IN THE
SOCIAL SECURITY ADMINISTRATION.
THERE WAS ONE GENTLEMAN BY THE
NAME OF GERALD RAY, WHO WAS THE
HEAD OF THE APPEALS COUNCIL
THERE AND HE WAS VERY EARLY ON
IN THE SPACE AND HE RECOGNIZED
THAT ALL WE WERE DOING AT THIS
AGENCY AT THE AT SOME 20 YEARS
BACK, WAS WRITING A BUNCH OF
DOCUMENTS AND WE WERE CAPTURING
NONE OF THE DATA THAT WOULD
ACTUALLY IMPROVE A SYSTEM THAT
HAS BEEN SORT OF RIFE WITH
DELAYS THAT ADJUDICATE SOME HALF
MILLION KIND OF CLAIMS ANNUALLY
FOR DISABILITY BENEFITS, AND HE
FIGURED OUT HOW TO BUILD OUT THE
DATA AND SORT OF INFRASTRUCTURE.
CAPTURE THAT INFORMATION. BRING
ON TALENT, RIGHT? AND HE DID
THAT, IN SPITE OF ALL OF THE
CHALLENGES THAT EXISTED AT THAT
POINT OF TIME, SO MUCH SO THAT
COLLEAGUES REFERRED TO HIM AS
THE STEVE JOBS OF THE SSA RIGHT,
IT SHOULD NOT REQUIRE A STEVE
JOBS. WE NEED TO FIGURE OUT HOW
TO LEARN FROM THOSE KINDS OF
INNO. USE CASES TO LEVEL THOSE
KINDS OF BARRIERS AND GET MORE
OF THAT KIND OF RESPONSIBLE
INNOVATION WITHIN OUR AGENCIES
AND TO ATTRACT THAT TALENT IN
SENATOR RUBIO. THANK YOU. GOOD
MORNING. UM YOU KNOW, I HAVE A
QUESTION IN REGARDS TO SOME OF
THE INFORMATION YOU JUST SHARED.
YOU TALKED ABOUT A I RESEARCH
HUBS AND HOW TO CREATE EXPERTS
IN THIS FIELD AND PERHAPS HAVE
ROTATIONS TO WORK IN THE CIVIL
SERVICE SORT OF MODEL, AND IT
MAY BE PREMATURE TO ASK YOU THIS
, BUT, UH, THE REASON WE'RE SORT
OF HERE. ONE OTHER ISSUES WE ARE
SEEING IS JUST THE DANGERS THAT
A I POSES. UM HOW ARE WE
THINKING AHEAD IN TERMS OF
CREATING THESE EXPERTS WITH
EXPERTISE THAT ARE GONNA BE IN
OUR, UM YOU KNOW, CIVIL SERVICE.
HOW DO WE SAFEGUARD THIS
INFORMATION FROM ALL OF A SUDDEN
YOU HAVE THESE INDIVIDUALS THAT
BECOME EXPERTS AND THEY LEAVE
THE FIELD. HOW ARE WE THINKING
OF SAFEGUARDING THE INFORMATION
BECAUSE THIS IS NOT LIKE A
FLOPPY DISK WHERE YOU CAN JUST
COPY AND LEAVE. NOW YOU HAVE THE
INFORMATION WHICH INTELLECTUAL
PROPERTY THAT YOU CANNOT PREVENT
SOMEONE FROM TAKING WITH THEM.
HOW ARE WE THINKING AHEAD, AND I
KNOW IT MAY BE PREMATURE TO ASK
YOU THIS QUESTION, BUT IT SCARES
ME TO THINK WE'RE CREATING
EXPERTS. CAN JUST LEAVE AT ANY
TIME AND THE INFORMATION LEAVES
WITH THEM. YEAH UH, IT UH, THANK
YOU, UH, SENATOR RUBIO FOR THAT
QUESTION WITHIN THE FEDERAL
GOVERNMENT, THERE IS AN ACTUAL
KIND OF OPEN SOURCE POLICY THAT,
UH, SORT OF DIRECT AGENCIES TO
ACTUALLY HAVE OPEN SOURCE
STANDARDS SO THAT WHEN YOU'RE
GOING THROUGH PROCUREMENT SYSTEM
, YOU'RE NOT SIMPLY SORT OF
OUTSOURCING THAT EXPERTISE AND
THEN YOU KNOW, PRIVATE VENDOR
MAY RUN AWAY WITH THAT AND TRY
TO SELL IT BACK TO YOU. AND SO I
THINK, UH, SETTING THOSE KINDS
OF STANDARDS, I THINK IS GONNA
BE REALLY IMPORTANT. NOW I THINK
ACTUALLY THE SORT OF A I
RESEARCH HUB IDEA, AND THIS KIND
OF A TALENT EXCHANGE, ACTUALLY,
IN MY VIEW REALLY HELPS BUILD
THE MUSCLES FOR RESPONSIBLE
INNOVATION. LET ME GIVE YOU ONE
STORY OF A PRESIDENTIAL
INNOVATION FELLOW WHO DID A
ROTATION WITH A FEDERAL AGENCY
AND WAS IN A KIND OF PROCUREMENT
SETTING WHERE A DEPARTMENT IS
BEING SOLD SOMETHING AND SORRY
IF I'M NOT GIVING YOU DETAILS
HERE, BUT BUT BUT THERE WAS A
VENDOR WHO WAS MAKING ALL SORTS
OF CLAIMS AND IT WAS THE
PRESIDENTIAL INNOVATION FELLOW
WHO ACTUALLY HAD SOME
ENGINEERING EXPERIENCE AND SAID,
LISTEN, YOU ARE JUST TELLING ME
STUFF THAT'S OFF OF YOUR WEBSITE
. THAT SORT OF, UH, MORE SOUNDS
MORE IN THE VEIN OF PR. I WANT
TO TALK TO YOUR ENGINEERS TO
ACTUALLY SEE WHAT SAFEGUARDS
HAVE BEEN TAKEN. AND THAT REALLY
LED THE AGENCY TO MAKE A MUCH
MORE INFORMED DECISION TO KNOW
HOW TO ASK THE RIGHT QUESTIONS
AND TO FIGURE OUT WHETHER OR NOT
THIS WAS ACTUALLY WORTH
PROCURING. SO ACTUALLY THINK
THESE TALENT EXCHANGES ARE
EXACTLY A WAY TO BUILD INTERNAL
CAPACITY AND TALENT. UH SO THAT
WE DON'T SEE FURTHER HOLLOWING
OUT OF THIS KIND OF CAPACITY.
YEAH. II I VERY MUCH AGREE IN
THAT. UM UH, STRENGTHENING THE
PUBLIC SECTOR. YOU YOU BUILDING
AS MUCH OF THIS EXPERTISE WITHIN
THE PUBLIC SECTOR AS POSSIBLE IS
A GOOD IS A GOOD ANSWER TO THAT,
UM I ALSO ACKNOWLEDGE THAT THE
RISK THAT YOU NOTICE THAT YOU
NOT A REAL, PARTICULARLY WHERE
THE SYSTEMS ARE USED FOR
REGULATORY ENFORCEMENT PURPOSES,
I THINK I THINK IT WAS YOUR
PROFESSOR HOS CO AUTHOR
PUBLICATION IN 20. 20 I WANT TO
SEE THE ONE THAT WAS DONE WITH
WITH A WITH KATHERINE SHARKEY
AND THE OTHERS. THAT MENTIONED
2018 RIGHT THAT THAT MENTIONED
UM, FOR EXAMPLE, USE OF THESE
TOOLS FOR S¼ E èC ENFORCEMENT
UM, A POTENTIALLY SUSPICIOUS
TRADING PRACTICES THAT NEEDED TO
BE INVESTIGATED. IF THE PERSON
WHO'S DESIGNING THOSE TOOLS CAN
THEN TURN AROUND AND SELL THEIR
SERVICES TO, YOU KNOW MAJOR
BANKS THAT ARE LOOKING TO AVOID
REGULATORY SCRUTINY THAT CREATES
A REALLY DANGEROUS SITUATION.
I'M NOT AN EXPERT IN IN IN SORT
OF CONFLICT OF INTEREST RULES OR
OR COOLING OFF PERIODS, AND THAT
THAT'S THAT'S LESS MY AREA. UM
BUT CERTAINLY I THINK THAT IT'S
IMPORTANT TO NOTE THE RISK,
ESPECIALLY AS THESE TOOLS ARE
EMPLOYED FOR REGULATORY
ENFORCEMENT. AND I JUST WANT TO
ASK YOU THAT, YOU KNOW, I
REMEMBER WHEN JUST TECHNOLOGY
COMPUTERS CAME. AND I REMEMBER
BEING A COUNCIL MEMBER WHERE OUR
POLICE DEPARTMENT WAS HELD
HOSTAGE ONE OF OUR POLICE
DEPARTMENTS, AND THEY LITERALLY
HAD TO BUY THE INFORMATION BACK
MILLIONS OF DOLLARS BECAUSE
SOMEONE FIGURED OUT HOW TO HOLD
THEM HOSTAGE, AND I'M THINKING
ALONG THOSE LINES. LIKE WHAT?
DON'T WE KNOW NOW? THAT MAY
CAUSE THIS KIND OF CONCERN IN
THE FUTURE, BUT YOU KNOW, AGAIN,
THANK YOU FOR YOUR EXPLANATION.
AND I THINK THAT YOU GAVE ME
ENOUGH. THANK YOU. LET ME ADD TO
ADD TO THAT. I MEAN, I. I THINK
IT'S ABSOLUTELY RIGHT THAT WE
HAVE TO THINK ABOUT THE RIGHT
STANDARDS FOR GOVERNMENT TO
ACTUALLY CONTINUE TO OWN THIS
DATA PART OF WHAT WE'RE SEEING
IN THE TURN TO GENERATIVE. A. I
IS THAT THEIR SERVICE IS BEING
SOLD. AND THEN THERE ARE SOME
VENDORS WHO WANT TO CLAIM THAT
DATA AS THEIR OWN AND THEN SELL
IT BACK TO GOVERNMENT AGENCIES.
THAT'S EXACTLY I THINK THE
SITUATION YOU'RE DESCRIBING OF
BEING HELD HOSTAGE, WHICH I
THINK WE VERY MUCH WANT TO AVOID
JUST TO GIVE YOU THE OTHER KIND
OF EXAMPLE OF WHY IN CAPACITY IS
SO IMPORTANT WHEN THE SECURITY
EXCHANGE COMMISSION HAD ITS OWN
INNOVATION HUB TO FIGURE OUT HOW
TO SIFT THROUGH ALL OF THE
FILINGS THAT IT RECEIVES TO SPOT
SIGNS FOR INSIDER TRADING. UM,
THERE WERE A BUNCH OF TECHNICAL
MACHINE LEARNERS WHO WERE
BUILDING OUT RISK MODELS FOR THE
RISK OF, SAY, INSIDER TRADING.
BUT IT WAS THE LINE LEVEL
ATTORNEYS WHO SAID I DON'T CARE
WHAT THE RISK SCORE IS. YOU HAVE
TO TELL ME WHY, UH, THIS RISK IS
BEING ACTIVATED CAUSE I
ULTIMATELY HAVE TO DEFEND THIS
IN FRONT OF A JUDGE, AND IT WAS
REALLY THAT INTERNAL KIND OF
CAPACITY THAT LED TO THE SORT OF
DUE PROCESS PROTECTIONS THAT WE
REALLY NEED SO THAT IT'S NOT
JUST BASED ON A KIND OF
UNARTICULATED BLACK BOX RISK
SCORE. UH BUT, UH, YOU KNOW,
SOMETHING THAT IS REALLY
EXPLAINED, AND HENCE AMENABLE TO
ACTUALLY BEING INVESTIGATED BY
LINE LEVEL ATTORNEY AND THANK
YOU, SENATOR, SENATOR HUMBER.
THANK YOU, SENATOR PADILLA. TWO
OBSERVATIONS, THEN ONE QUESTION.
UM, OBSERVATION. ONE IS THAT IN
THE EARLY TWO THOUSANDS THE
STATE WITH COMPTROLLER STEVE
WESLEY, I THINK AT THE HELM OF
THE CONTROLLER'S OFFICE DECIDED
WE WERE GOING TO REVAMP OUR HÌ R
PAYROLL SYSTEM THAT THAT HASN'T
BEEN DONE. IT'S 2024 I. I WAS
INVOLVED IN THREE YEARS, MAYBE
FOUR YEARS OF LITIGATION WITH S¼
A P. WHAT WAS VERY CLEAR IS THAT
THE STATE WAS OUTMATCHED BY THE
EXPERTISE ON THE OTHER SIDE. WE
SUBSEQUENTLY SETTLED FOR 80
THREE MILLION DOLLARS, BUT IT
WAS VERY, VERY CLEAR THAT THOSE
ON THE OTHER SIDE OF THE TABLE
MAKING THREE TIMES WHAT STATE
EMPLOYEES WERE MAKING AT THE
TIME WITH MANY MORE YEARS OF
EXPERIENCE, REALLY ATE OUR LUNCH
. UH, THAT HASN'T BEEN RESOLVED
SO FAR AS I CAN TELL. ONE SYSTEM
IS STILL A PROBLEM. BUT TWO MORE
IMPORTANTLY, IS THAT IS THAT WE
HAVEN'T OVERCOME THAT CHALLENGE
TO MAKE SURE THAT THE FOLKS WHO
ARE NEGOTIATING ON BEHALF OF THE
PROCUREMENT ISSUES ARE AT LEAST
SOMEWHERE IN THE SAME RANGE OF
COMPENSATION AS THOSE ON THE
OTHER SIDE OF THE TABLE, AND
THAT NEEDS TO BE FIXED. SECONDLY
IS THAT WE HAVE OUR OWN
CHALLENGE HERE INTERNALLY AS I
MENTIONED AT THE OUTSET A I
LEGISLATION IS ALL COMING TO THE
JUDICIARY COMMITTEE. WE HAVE
INCREDIBLY SMART LAWYERS ON
JUDICIARY COMMITTEE HAVE DONE
EXCELLENT JOB. UM BUT BUT THIS
IS AT LEAST FROM MY PERSPECTIVE,
HARD STUFF. AND WE INTERNALLY
NEED TO HAVE OUR OWN EXPERTISE
INTERNALLY TO BE ABLE TO, UM.
MAKE JUDGMENTS WITHOUT A
PARTICULAR BIAS. SO
NOTWITHSTANDING, UM YOU KNOW, U
èC L, A AND STANFORD'S
ACADEMIC, UM BASICALLY STANDING
IN THE WORLD, UM, RELYING ON
BASICALLY ANYBODY OUTSIDE, UH,
SUBJECTS US TO OUR OWN BIAS, AND
SO WE NEED TO GET OVER THAT, AND
WE NEED TO HAVE OUR OWN
TECHNOLOGICAL EXPERTISE. I THINK
WITHIN COMMITTEES THIRD
OBSERVATION IS IN A QUESTION IS,
UH, WITHIN THE DEPARTMENT OF
TECHNOLOGY, DO YOU THINK WE NEED
TO HAVE A, UM, A I DEPARTMENT
WITHIN THE DEPARTMENT OF
TECHNOLOGY HERE IN CALIFORNIA.
OH, GO AHEAD. YEAH PLEASE. THANK
YOU, SENATOR UMBERG FOR THAT
QUESTION, I THINK YOUR ENTIRELY
RIGHT THAT BUILDING THAT
INTERNAL CAPACITY IS GOING TO BE
ABSOLUTELY CRITICAL,
PARTICULARLY IN THE PROCUREMENT
SETTING. I I CHAIRED THE
NATIONAL A. I ADVISORY
COMMITTEE'S WORKING GROUP ON
PROCUREMENT. AND WHEN WE DID A
SERIES OF INTERVIEWS, ONE OF THE
REAL CHALLENGES HAS BEEN THE
ALLOCATION OF RESPONSIBILITY
BETWEEN PROCUREMENT OFFICIALS
AND BUSINESS UNITS. SO, FOR
INSTANCE, IN SOME AGENCIES. THE
PROCUREMENT OFFICIALS WOULD SAY,
WELL, RESPONSIBLE A I THAT'S
ENTIRELY UP TO THE BUSINESS
UNIT, AND WE'LL JUST WRITE THE
CONTRACT AND WORK IT THROUGH THE
PROCUREMENT SYSTEM, AND THE
BUSINESS UNIT WILL SAY, WELL, NÓ
NÓ. THAT'S THAT'S RESPONSIBLY.
YEAH, THAT THAT SHOULD BE UP TO
THE PROCUREMENT OFFICIALS. AND
THE REALITY IS THAT WE'RE NOT AT
A STATE WHERE YOU CAN EX ANTE.
SPECIFY ALL OF THIS. A I SYSTEMS
ADAPT OVER TIME, SO COMPUTER
VISION SYSTEM THAT'S TRAINED ON
A HOSPITAL PATIENT POPULATION IN
YEAR ONE AS THE PATIENT
POPULATION CHANGES WILL DE IN
PERFORMANCE, AND SO WE NEED
BETTER FORMS OF KIND OF
MONITORING OVER TIME. THAT AT
LEAST IN SOME PROCUREMENT RULES
ARE JUST NOT OR A LITTLE BIT
MORE FOREIGN TO KIND OF HAVE
THAT KIND OF QUALITY CONTROL
GOING FORWARD, SO I THINK I 100%
AGREE WITH YOU ABOUT THE NEED
FOR INTERNAL PROCUREMENT
EXPERTISE. THAT'S WHY I'M SUCH A
FAN OF, UH, THE PROCUREMENT
INNOVATION LAB AT THE DEPARTMENT
OF HOMELAND SECURITY THAT IS
REALLY TRIED TO MODERNIZE THESE
KINDS OF SYSTEMS AS TO YOUR
QUESTION ON THE TECHNOLOGY. I
DON'T KNOW ENOUGH AS TO WHERE
THE EXACT RIGHT LOCATION OF
BUILDING KIND OF MORE A I
EXPERTISE. UH UH, IS WITHIN THE
WITHIN THE FEDERAL GOVERNMENT?
WE STOOD UP AN A I CENTER FOR
EXCELLENCE UNDER THE A. I N
GOVERNMENT ACT THAT SITS UNDER
THE GENERAL SERVICES
ADMINISTRATION. I THINK ONE
THING I WOULD BE MINDFUL OF. IS
THAT GOING BACK TO WHAT I SAID
WITH THE NATIONAL SECURITY
COMMISSION ON A I WE HAVE TO
FIGURE OUT TO ACTUALLY ATTRACT
TECHNICAL TALENT AND GIVE THEM
MEANINGFUL THINGS TO WORK ON,
AND THAT DOES NOT NECESSARILY
MEAN THAT WE SORT OF HOUSE THEM
IN THE WAY WE HAVE
CONVENTIONALLY THOUGHT ABOUT
INFORMATION TECHNOLOGY SERVICES.
AND SO I THINK THAT'S ONE OF THE
INTERESTING INSTITUTIONAL
CHALLENGES OF WHERE SOMETHING
LIKE THIS. UH UH, SITS
APPROPRIATELY. YEAH I ABSOLUTELY
THINK THAT IT IS IMPORTANT THAT
THE DEPARTMENT OF TECHNOLOGY HAS
A I EXPERTISE AND AN APPROPRIATE
. CAPACITY AND MANDATE TO FOCUS
ON THESE ISSUES AS TO WHETHER
THAT SHOULD BE STOOD UP AS AN
INDEPENDENT UNIT WITHIN THAT. I
MEAN, YOU KNOW, THESE
TECHNOLOGIES ARE TRANSFORMATIVE,
AND THEY'RE GOING TO BE
EVERYWHERE, SO I THINK IT'S.
IT'S AN INTERESTING QUESTION AS
TO WHETHER OR NOT AND I AND I'M
NOT I'M NOT INTIMATELY FAMILIAR
ENOUGH WITH WITH THE STRUCTURE
OF THAT DEPARTMENT. UM BUT YOU
KNOW IT. IT STRIKES ME THAT, UM
IT'S LESS A QUESTION OF
UNDERSTANDING THAT THE SPECIFICS
OF THIS TECHNO THIS TECHNOLOGY
AS AS IT'S TRANSFORMATIVE IMPACT
EVERYWHERE, AND I FEEL LIKE
THAT'S GONNA TOUCH EVERY ASPECT
OF THEIR MANDATE. REGARDLESS UM,
I DEFINITELY AGREE THAT THERE
NEEDS TO BE A COORDINATING
AGENCY. AROUND UH, TRACKING AND
ASSESSING AND PROVIDING
EXPERTISE AROUND PUBLIC USES OF
A I, UM, WHETHER THE MOST
APPROPRIATE PLACE FOR THAT IS TO
HOUSE IT WITHIN THE DEPARTMENT
OF TECHNOLOGY IS A SEPARATE
QUESTION. AND AGAIN. I'M NOT. I
DON'T KNOW THAT I'M INTIMATE
ENOUGH WITH WITH WITH WITH HOW
THAT, UM, WITH THE EXPERTISE
WITHIN THAT INSTITUTE TO KNOW IF
THAT'S THE APPROPRIATE HOME FOR
IT, BUT I DEFINITELY AGREE THAT
COORDINATING THAT THERE IS A
NEED FOR A HUB FOR EXPERTISE AND
A COORDINATING AGENCY. IF I
COULD SAY TWO MORE COMMENTS ON
THE REALLY IMPORTANT QUESTION,
PROFESSOR HO IF I MIGHT EXCUSE
THE INTERRUPTION, IF YOU COULD
BE BRIEF IN YOUR RESPONSES, WE
HAVE TWO MORE PANELS TO GET TO,
AND PROBABLY ADDITIONAL
QUESTIONS FROM MEMBERS WITH
REGARD TO THEIR PRESENTATION.
VERY GRATEFUL FOR YOUR
PARTICIPATION. BUT IF YOU COULD
BE BRIEF IN ADDRESSING THE
SENATOR'S QUESTION OR FOLLOWING
UP, AND WE'LL MOVE SHORT, I
APOLOGIZE. I'LL JUST VERY
QUICKLY. I THINK IT DEPENDS ON
THE FUNCTIONS AS WELL. SO THE
CYBER SECURITY INTEGRATION
CENTER MIGHT BE A BETTER PLACE
TO LOCATE SOMETHING LIKE ADVERSE
EVENT REPORTING. THEY ARE
ALREADY HAVE THE SORT OF
INFRASTRUCTURE TO RECEIVE
REPORTS ABOUT VULNERABILITIES
AND ATTACKS. YOU ANY ADDITIONAL
MEMBERS WANT TO QUESTION THIS
PANEL, SENATOR WEINER WELCOME.
MISTER PRESIDENT. I WANT TO
THANK BOTH CHAIRS FOR CONVENING
THIS IMPORTANT HEARING TODAY.
THANK YOU FOR COMING IN FOR YOUR
WORK. UM YOU KNOW, I, I, UM I
WILL SAY PROUDLY THAT I, UM I UH
UM, I BELIEVE, AND I THINK I'M
RIGHT. I REPRESENT THE HEARTLAND
OF A I INNOVATION IN SAN
FRANCISCO. UH AND, UM, UM, YOU
KNOW, VERY PROUD OF THAT. I
THINK A I HAS INCREDIBLE
POTENTIAL TO MAKE THE WORLD A
BETTER PLACE TO I MAKE LIFE
BETTER FOR HUMANITY ON SO MANY
LEVELS. WE ALSO KNOW THAT THERE
ARE REAL RISKS, AND I THINK IT'S
REALLY IMPORTANT NOT TO BE
ALARMIST BECAUSE BEING ALARMIST
COULD RESULT IN CHOKING OFF
INNOVATION, AND THAT'S THE LAST
THING WE WANT TO DO. WE WANT TO
BE PRO INNOVATION. UM BUT I ALSO
IT'S IMPORTANT NOT TO BE
DENIALIST ABOUT THE VERY REAL
RISKS THAT WE NEED TO GET AHEAD
OF, AND THAT'S REALLY, UM IN MY
VIEW. WHAT WE NEED TO GET AHEAD
OF THE RISKS. WE HAVE A LONG
HISTORY, NOT JUST IN CALIFORNIA.
BUT ON THIS IN THE U S¼ AND ON
THIS PLANET OF NOT GETTING AHEAD
OF RISKS CREATED BY NEW
TECHNOLOGY, PARTIALLY FOR FEAR
OF STIFLING INNOVATION, BUT
PARTIALLY BECAUSE I THINK
SOMETIMES MAKERS ARE GINGER
ABOUT GETTING INVOLVED AND NOT
UNDERSTANDING THE IMPLICATIONS
AND ALSO UNDERSTANDING THAT THE
LEGISLATIVE PROCESS IS NOT
ALWAYS THE MOST NIMBLE ONE,
WHICH IS WHY IT'S SO IMPORTANT
TO HAVE ADMINISTRATIVE CAPACITY
IN ORDER TO BE NIMBLE BECAUSE
TECHNOLOGY CHANGES SO QUICKLY
AND WE WANT TO MAKE SURE THAT WE
CAN, UH, CHANGE WITH THAT. BUT
THAT DOESN'T MEAN THAT WE SHOULD
JUST THROW UP OUR HANDS. WE'VE
DONE THAT WITH SOME TECHNOLOGY
ISSUES IN THE PAST AND HAVE
WHICH HAS RESULTED IN SOME VERY,
VERY BAD OUTCOMES AROUND PRIVACY
AROUND SOCIAL MEDIA, UH AND SO
FORTH, AND THEN 10 YEARS LATER,
WE FINALLY GET AROUND TO IT, AND
IT'S TOO LATE TO DEAL WITH THE
SEVERE HARMS THAT HAVE RESULTED
AND SO WE WHEN IT COMES TO A I
AND WE NEED TO WORK VERY HARD TO
NOT JUST ALLOW INNOVATION TO
HAPPEN, BUT ACTUALLY FOSTER
INNOVATION AND DO EVERYTHING WE
CAN TO SUPPORT INNOVATION, BUT
WE SHOULD NOT STICK OUR HEADS IN
THE SAND AND PRETEND THAT THERE
AREN'T VERY, VERY REAL RISKS AND
WE HAVE A RESPONSIBILITY AS
GOVERNMENT TO GET AHEAD OF THOSE
RISKS WORKING COLLABORATIVELY
WITH THE PEOPLE WHO ARE
INNOVATING. UH BUT WE SHOULD NOT
PRETEND THAT THOSE RISKS DON'T
EXIST BECAUSE THERE IS THERE'S
GOING TO BE A REAL TENDENCY AND
I KNOW WE'RE GOING TO A LOT OF
VOICES SAYING, DON'T WORRY
NOTHING TO SEE HERE. THERE IS
SOMETHING TO SEE HERE AND WE
OUGHT TO TAKE IT SERIOUSLY. SO
AGAIN TO BOTH CHAIRS. THANK YOU
SO MUCH FOR CONVENING THE
HEARING TODAY. THANK YOU. THANK
YOU. I APPRECIATE THAT. YOU WANT
TO BRIEFLY RESPOND TO THAT, AND
THEN I'M GOING TO INVITE OUR
SECOND PANEL. THANK YOU, SENATOR
WEINER AND THANK YOU FOR YOUR
LEADERSHIP ON THESE ISSUES. I
CAN AGREE MORE. WE HAVE TO GET
AHEAD OF THESE RISKS. I THINK WE
ALSO NEED TO BE SURE THAT
GOVERNMENT HAS THE APPROPRIATE
INFORMATION ON WHAT RISKS TO
ACTUALLY ADDRESS. THAT'S WHY I'M
SUCH A FAN OF ADVERSE EVENT
REPORTING. UH, BECAUSE, UH, AS I
NOTED IN MY OPENING REMARKS, WE
HAD MANY MONTHS ON CAPITOL HILL,
WHERE THERE WAS A LOT OF
CONSTERNATION AROUND THE BIO
WEAPONS RISK, AND AT LEAST THE
CONSENSUS NOW HAS BEEN THROUGH
THE BEST DESIGN. THAT ACTUALLY,
AT LEAST GIVEN CURRENT
TECHNOLOGY. THAT RISK IS NOWHERE
ANYWHERE. WHAT PEOPLE HAD KIND
OF GINNED UP. UM AND SO I THINK
THAT'S WHY, UH, KIND OF ADVERSE
EVENT REPORTING IS A REALLY
IMPORTANT WAY IN WHICH
GOVERNMENT CAN INFORM ITSELF.
LET ME SAY ONE OTHER THING,
WHICH IS THAT THE THING I AM
WORRIED ABOUT IS FORMS OF
REGULATION THAT REALLY HAVE
ADVERSE KIND OF COMPLIANCE
COSTS. PARTICULARLY BECAUSE THE
HISTORY OF CYBER SECURITY IS ONE
WHERE OPEN INQUIRY AND BEING
ABLE TO INTERROGATE THE
WEAKNESSES OF MODELS HAS BEEN SO
IMPORTANT, AND THERE ARE SOME
PROPOSALS OUT THERE THAT COULD
REALLY MAKE IT MUCH HARDER. FOR
INSTANCE, FOR ACADEMICS, PUBLIC
INTEREST GROUPS REALLY, TO
INTERROGATE THE WEAKNESSES OF
THESE MODELS, AND WE HAVE TO
DESIGN REGULATIONS THAT DON'T
HAVE THAT KIND OF ADVERSE EFFECT
. AN OPEN SOURCE IS DEFINITELY A
TOPIC OF INTENSE. AND THEIR
FOLKS, OF COURSE WHO DON'T AGREE
WITH YOU, BUT I ALSO IT'S NOT
JUST ABOUT CURRENT TECHNOLOGY.
IT'S ABOUT THE TECH BECAUSE WE
KNOW THAT IN THE NEXT YEAR OR
TWO YEARS, FIVE YEARS, UM, A. I
IS NOT GOING TO LOOK LIKE IT IS
TODAY. IT'S GOING TO BE
DRAMATICALLY MORE POWERFUL
MODELS. AND SO WE WANNA JUST GET
AHEAD OF THAT, SO WE'RE AND
AGAIN, NOT STOPPING IT.
HAPPENING WE WANT THE INNOVATION
TO PROCEED. BUT IF WE'RE
CONTINUALLY PLAYING CATCH UP, UM
, THAT THAT JUST DOESN'T WORK
VERY WELL. SO THANK YOU. THANKS
TO YOU ALL IN THE INTEREST OF
TIME. I'M GONNA WRAP I'M GONNA
THANK OUR DISTINGUISHED
PANELISTS FOR A PANEL NUMBER
ONE. PROFESSOR DIRECTOR. THANK
YOU VERY MUCH. IT'S MY HONOR TO
WELCOME UP OUR SECOND PANEL, UH,
WHICH WILL FOCUS ON WORKFORCE
AND THE INNOVATIVE PERSPECTIVE.
ADDIE COOK IS GLOBAL A I PUBLIC
POLICY LEAD AT GOOGLE CLOUD AND
ANNETTE BERNHARDT IS DIRECTOR OF
TECHNOLOGY AND WORK PROGRAM. AT
THE UNIVERSITY OF CALIFORNIA,
BERKELEY INSTITUTE FOR RESEARCH
ON LABOR AND EMPLOYMENT. PLEASE
MAKE YOURSELVES COMFORTABLE.
WELCOME THANKS FOR YOUR
PARTICIPATION. UH AND EDDIE,
PLEASE PROCEED WHEN READY. GOOD
MORNING, CHAIRS PADILLA, DODD.
WIENER AND MEMBERS OF THE
COMMITTEE. THANK YOU FOR THE
OPPORTUNITY TO TESTIFY TODAY ON
THE USE OF ARTIFICIAL
INTELLIGENCE IN STATE GOVERNMENT
AND WORKFORCE. MY NAME IS ADDIE
COOK, AND I AM THE GLOBAL A. I
PUBLIC POLICY LEAD AT GOOGLE
CLOUD. I'M EXCITED TO BE BACK IN
CALIFORNIA, WHERE I BEGAN MY
POST UNIVERSITY PUBLIC SERVICE
CAREER AS A MEMBER OF AMERICORP
VISTA SERVING YOUTH IN LOS
ANGELES PUBLIC HOUSING TO THIS
DAY. THAT EXPERIENCE CONTINUES
TO INFORM MY POLICY WORK IN A I
IN MY ROLE AT GOOGLE. I WEAR A
FEW DIFFERENT HATS. AS MY TITLE
SUGGESTS. I SUPPORT OUR
LEGISLATIVE ENGAGEMENTS AROUND
THE GLOBE, INCLUDING IN THE EU.
BUT MY ROLE REALLY IS TO SUPPORT
OUR PRODUCT TEAMS AND OUR
CUSTOMERS WHO USE OUR PRODUCTS,
INCLUDING IN PUBLIC SECTOR. WHEN
I BEGAN THIS ROLE THREE YEARS
AGO, WAS GIVEN THE OPPORTUNITY
TO SERVE AS A REPRESENTATIVE ON
OUR MANAGER LEVEL. A I
PRINCIPALS COMMITTEE. GOOGLE
BEGAN DEVELOPING THESE
PRINCIPLES TO GUIDE THE
DEVELOPMENT AND USE OF A I
RESULTING IN THEIR PUBLICATION
IN 2018. SINCE THEN WE HAVE BEEN
WORKING TO IMPROVE
IMPLEMENTATION ORGANIZATION AND
GOVERNANCE OF OUR A. I
PRINCIPLES ACROSS GOOGLE AND IN
GOOGLE CLOUD. OVER THE PAST YEAR
, I HAVE HELPED ADVISE AND GUIDE
UPDATES TO OUR PROCESS TO ASSURE
ENSURE ALIGNMENT WITH ALL THE
FORTHCOMING A. I REGULATIONS,
STANDARDS AND FRAMEWORKS,
INCLUDING IN PUBLIC SECTOR. WE
PROUDLY CONTRIBUTED TO THE
NATIONAL INSTITUTE OF STANDARDS
AND TECHNOLOGIES. A. I RISK
MANAGEMENT FRAMEWORK WITHIN
GOOGLE. OUR GOVERNANCE PROGRAM
IS ALIGNED WITH THE A IRMF
APPROACH. AND UNDERPINNED BY
INDUSTRY, LEADING RESEARCH AND A
GROWING LIBRARY OF RESOURCES
THAT WE MAKE AVAILABLE TO THE
PUBLIC. IMPORTANTLY THE NEST
APPROACH, WHICH HAS BEEN
SUCCESSFUL IN CYBER SECURITY,
AND PRIVACY, IS FLEXIBLE AND CAN
ADAPT AS THE ECOSYSTEM
PROGRESSES. THE A IMF PROVIDES
GUIDANCE TO DEVELOPERS AND
EMPLOYERS OF A I SYSTEMS ON HOW
TO STRIKE A PRACTICAL BALANCE
BETWEEN OPTIMIZING BENEFICIAL
USE CASES IN ADDRESSING
POTENTIAL RISKS. WE LOOK FORWARD
TO OUR CONTINUED WORK WITH NS
AND THE MANY STATES WHO ARE
CONSIDERING ADOPTING NIST AS
THEIR FOUNDATION. AS THE A I
ECOSYSTEM MATURES, NEW
TECHNIQUES AND APPLICATIONS ARE
DEVELOPED AND FURTHER PROGRESS
IS MADE. FRAMEWORKS LIKE THIS
CAN ENSURE THAT WE ARE ALL
ALIGNED ACROSS 50 STATES.
ALIGNMENT IS IMPORTANT,
ESPECIALLY IN THE DELIVERY OF
GOVERNMENT SERVICES. WHETHER
IT'S ADMINISTERING UNEMPLOYMENT
BENEFITS MORE EFFICIENTLY OR
PROVIDING COMMUNITIES WITH
AUTHORITATIVE INFORMATION ABOUT
VACCINES. THESE PRACTICAL
APPLICATIONS OF A I WROTE WORK
AND DELIVER BETTER OUTCOMES FOR
THE PUBLIC AT LARGE. A I CAN
SUPPORT AGENCIES BY PROCESSING
PAPERWORK, DIGITIZING CLAIMS
REQUESTS AND AUTOMATING THESE
ROAD TASKS SUCH AS REVIEWING
APPLICATIONS, THE VERY THINGS
THAT SLOW SERVICE DELIVERY AND
LEAD TO CIVIL SERVANT BURNOUT.
USING PRACTICAL AND RESULTS
ORIENTED A. I ALSO MEANS MORE
SATISFIED CUSTOMERS AND COST
EFFECTIVE SERVICE DELIVERY. WITH
LESS GROWTH WORK. GOVERNMENT
EMPLOYEES CAN EXPECT MORE
CREATIVE AND SATISFYING
ENGAGEMENTS WITH BOTH THE PUBLIC
AND EACH OTHER. THESE ARE
CRITICAL POSITIVE OUTCOMES FOR
DELIVERING GOVERNMENT SERVICE.
JUST THIS PAST WEEK. GOOGLE IS
PROUD TO SUPPORT A GREAT EXAMPLE
OF THIS WORK. WE SUPPORTED THE
NEW YORK STATE DEPARTMENT OF
LABOR AND WHO WAS RECOGNIZED
LAST WEEK FOR THEIR INNOVATION
AND INFORMATION TECHNOLOGY FROM
THE NATIONAL ASSOCIATION OF
STATE WORKFORCE AGENCIES.
PERKINS THEIR INNOVATIVE WEB
BASED CHATBOT POWERED BY GOOGLE
CLOUD A. I CAN RESPOND TO 14
LANGUAGES 24 HOURS, SEVEN DAYS A
WEEK. THIS IS REALLY IMPORTANT
FOR WORKERS WHO WORK A
NONTRADITIONAL 9 TO 5 AND MAY
HAVE IMPORTANT QUESTIONS ABOUT
THEIR BENEFIT CLAIMS OUTSIDE OF
NORMAL WORKING HOURS FOR A STATE
GOVERNMENT AGENCY. SINCE
LAUNCHING IN AUGUST, 2021
PERKINS HAS HANDLED MORE THAN
100 AND 28,000 CLAIMS SPECIFIC
INQUIRIES WITH PERKINS.
CUSTOMERS SIMPLY SELECT THEIR
LANGUAGE AND ANSWER A COUPLE OF
PROMPTS ABOUT WHY THEY'RE
REACHING OUT. AND THEN ONCE
LOGGED IN, THEY CAN ASK SPECIFIC
QUESTIONS ABOUT THEIR IN ABOUT
THEIR INQUIRIES, FOR INSTANCE,
IF THE CUSTOMER BELIEVES THAT
THEY RECEIVED AN INCORRECT
PAYMENT AMOUNT AFTER SUBMITTING
THE BENEFIT IN QUESTION, PERKINS
WILL PROVIDE A BREAKDOWN OF THE
BENEFIT PAID, INCLUDING THE
DEDUCTIONS THAT WERE TAKEN FROM
THAT BENEFIT. FROM THERE, THE
CUSTOMER CAN GET MORE
INFORMATION ON EACH DEDUCTION
AND ELECT TO END THE CHAT. OR
THEY CAN ASK ADDITIONAL
QUESTIONS INTO THEIR ISSUE IS
RESOLVED. THIS ALSO MIGHT
INCLUDE ROUTING TO A HUMAN WE
UNDERSTAND THAT COMPUTER IS NOT
FOR EVERYONE WITH PERKINS. WE
HAVE ALSO WORKED WITH THE STATE
TO ENABLE VIRTUAL PHONE AGENTS
THAT DO THE SAME THING. THE
SYSTEM HAS BEEN UPDATED TO
UTILIZE SPEECH RECOGNITION TO
CAPTURE RESPONSES. SO YOU CAN
SAY THINGS LIKE HOW DO I FILE A
CLAIM? HOW DO I CHECK THE STATUS
OF MY PAYMENT? HOW DO I CERTIFY
FOR BENEFITS? SINCE JULY, 2023.
OVER THREE MILLION CONVERSATION
HAVE BEEN POWERED BY THIS
TECHNOLOGY GUIDING USERS THROUGH
THIS PROGRAM WITH EASE AND
FREEING UP NEW YORK CALL CENTER
REPRESENTATIVES TO HANDLE MORE
COMPLEX INQUIRIES FROM CITIZENS.
THE SYSTEM ALSO ALLOWS THE
DEPARTMENT TO CAREFULLY ANALYZE
INTERACTIONS WITH THEIR CONTACT
MANAGEMENT SYSTEM. BUT NOT IT.
THIS SYSTEM DOES NOT MAKE ANY
DECISIONS OR DETERMINATIONS.
THIS IS AN IMPORTANT RISK
MANAGEMENT DECISION THAT THE
DEPARTMENT OF LABOR MADE AND WE
WERE PROUD OF GOOGLE CLOUD TO
SUPPORT THAT, BUT ULTIMATELY THE
DECISION WAS MADE BY THE
DEPARTMENT. TODAY I'LL I'LL END
WITH THREE RECOMMENDATIONS FOR
CALIFORNIA GOVERNMENT AGENCIES.
NUMBER ONE. GOVERNANCE
STRUCTURES AND IMPLEMENTATION
MECHANISMS WITHIN AGENCIES
SHOULD BE ADEQUATELY RESOURCED.
AS THE LEGISLATURE. I KNOW THAT
A LOT OF MANDATES WILL COME OUT
BUT RESOURCING THE AGENCIES TO
RECEIVE THOSE MANDATES WILL BE
CRITICAL. WHEN AGENCIES ARE
TASKED WITH PROCUREMENT AND
OVERSIGHT. THEY NEED DEDICATED
RESOURCES TO ESTABLISH THEIR OWN
A I GOVERNANCE SYSTEMS AND
TRAINING FOR AGENCY OFFICIALS
ACROSS THE AGENCY CHARGED WITH
CARRYING OUT OVERSIGHT AND
PROCUREMENT. AGENCIES WILL
ULTIMATELY BEAR PRIMARY
RESPONSIBILITY FOR OVERSIGHT
BECAUSE ONLY THEY CAN VERIFY THE
END USES TO WHICH THEIR SYSTEMS
ARE BEING PUT AND ANY ADDITIONAL
TRAINING ANY ADDITIONAL DATA
THAT HAS BEEN INPUT INTO THEIR
TRAINING SYSTEM. THEY SHOULD
RECEIVE ROBUST TRAINING AND
RESOURCES TO EFFECTUATE THIS
RESPONSIBILITY. WHETHER THROUGH
THE USE OF INTERNAL FUNDING
DECISIONS OR GOVERNMENT WIDE
BUDGETING TO ENABLE REPEATABLE
SCALABLE, EFFICIENT AND
EFFECTIVE USE OF A I TO PRODUCE
OPTIMAL OUTCOMES FOR THE PUBLIC.
NUMBER TWO, AND I WAS GLAD TO
HEAR THIS QUESTION BE ASKED
EARLIER. AGENCIES SHOULD DEVELOP
A PRIVACY POLICY AND, FRANKLY,
THE ENTIRE GOVERNMENT OF
CALIFORNIA SHOULD HAVE A PRIVACY
POLICY THAT ENSURES EMPLOYEES
UNDERSTAND THE DIFFERENCE
BETWEEN COMMERCIAL AND
ENTERPRISE. A I OFFERINGS. THIS
IS CRITICAL TO THE SECURITY OF
THE SYSTEMS AS WELL. IN 2020.
GOOGLE CLOUD ANNOUNCED OUR
PUBLIC A IL DATA PRIVACY
COMMITMENT. IN THIS COMMITMENT.
WE PROMISE THAT BY DEFAULT,
GOOGLE DOES NOT USE CLOUD
CUSTOMER DATA FOR MODEL TRAINING
PURPOSES OR OTHER PRODUCT
IMPROVEMENT PURPOSES UNLESS A
CUSTOMER HAS OTHERWISE PROVIDED
WRITTEN PERMISSION TO DO SO. OR
HAS IT OPTED TO TERMS FOR SUCH
DATA USE? WE ARE. WE PROVIDE OUR
CUSTOMERS A FROZEN MODEL THAT
WILL NOT RETRAIN THE BASE MODEL.
UNLIKE OUR COMPETITORS, WE
EXTENDED THIS PROTECTION TO OUR
GENERATIVE A. I PRODUCTS FROM
THE VERY START. OUR COMMITMENT
TO ALL GOOGLE CLOUD CUSTOMERS.
INCLUDES YOUR DATA IS YOUR DATA,
YOUR PRIVACY IS PROTECTED. WE
WILL NOT AND CANNOT LOOK AT IT
WITHOUT A LEGITIMATE NEED TO
SUPPORT YOUR USE OF OUR SERVICE.
AND YOUR DATA DOES NOT RETRAIN
OUR MODELS. WE DON'T USE THAT
DATA. YOU YOU PROVIDE US TO
TRAIN OUR MODELS WITHOUT YOUR
PERMISSION. NUMBER THREE IN MY
FINAL RECOMMENDATION. PRIORITIZE
PORTABILITY AND
INTEROPERABILITY. AS MORE
GOVERNMENT DATA IS STORED IN THE
CLOUD. INTEROPERABILITY IS A
CRITICAL FACTOR FOR ENSURING
GOVERNMENT AGENCIES CAN
FACILITATE DATA PORTING AND
INTEROPERABILITY ACROSS MULTI
CLOUD ECOSYSTEMS. DATA
PORTABILITY AND INTEROPERABILITY
ARE CENTRAL TO INNOVATION AND
HELP BOOST COMPETITION. BETTER
DELIVERY OF GOVERNMENT SERVICE
FOR CALIFORNIANS REQUIRES BETTER
DATA COLLABORATION ACROSS
AGENCIES. GOOGLE HAS ALWAYS
BELIEVED ROBUST AND RECIPROCAL
PORTABILITY OFFERINGS SHOULD
REDUCE OR ELIMINATE ELIMINATE
SWITCHING COSTS. SWITCHING COSTS
ARE ASSOCIATED WHEN YOU CHANGE
FROM ONE SERVICE TO ANOTHER.
RESULTING IN MORE INNOVATIVE AND
USER FOCUSED PRODUCTS.
PORTABILITY AND OPERABILITY
MAKES IT EASIER FOR CUSTOMERS TO
CHOOSE AMONG SERVICES AND
FACILITATES COMPETITION WITHIN
CLOUD SERVICE SPACES. EQUALLY
WHEN IT COMES TO OUR ENTERPRISE,
CLOUD SERVICES, PORTABILITY,
CHOICE AND OPENNESS ARE AT THE
HEART OF OUR PHILOSOPHY AND
FUNDAMENTAL TO HOW WE DEVELOP
OUR TECHNOLOGY. JUST TODAY. IN
FACT, WE MADE A BIG ANNOUNCEMENT
ON OPEN SOURCE. SENATOR WEINER
TO FURTHER ADVANCE OUR
COMMITMENT TO OPENNESS. WE
ANNOUNCED OUR FIRST OPEN SOURCE
GENERATIVE A I MODEL CALLED
GEMMA, WHICH IS A WHICH SHE'S
USED BY THE SAME TECHNOLOGY THAT
WE USE FOR OUR GEMINI LINE OF
PRODUCTS. WE KNOW FROM PAST
TECHNOLOGICAL ADVANCES THAT IS
NECESSARY FOR KEY STAKEHOLDERS
TO COME TO THE TABLE WITH A
HEALTHY GRASP OF BOTH POTENTIAL
BENEFITS AND CHALLENGES. THANK
YOU FOR THE OPPORTUNITY FOR ME
TO SPEAK TODAY IN FRONT OF THE
COMMITTEE. I'M HAPPY TO ANSWER
ANY QUESTIONS YOU HAVE. THANK
YOU, MS. COOK DIRECTOR
BERNHARDT. MORNING CHAIRMAN DODD
, CHAIRMAN PADILLA AND SENATORS
AND THANK YOU FOR THE
OPPORTUNITY TO TESTIFY TODAY. MY
NAME IS ANNETTE BERNHARDT. I
DIRECT THE TECHNOLOGY AND WORK
PROGRAM AT THE UC BERKELEY LABOR
CENTER, AND I AM ALSO
CONTRIBUTING TO A SERIES OF
WORKSHOPS AT THE UC BERKELEY
DATA SCIENCE SCHOOL IS CONVENING
AROUND A I GOVERNANCE. I WANT TO
COMMEND YOUR LEADERSHIP ALONG
WITH THAT OF GOVERNOR NEWSOM AND
HIS HIS ADMINISTRATION IN MAKING
CALIFORNIA A NATIONAL LEADER IN
RESPONDING TO THE A. I
REVOLUTION. LIKE YOU AND OTHERS
HERE. I BELIEVE THE PUBLIC
SECTOR SHOULD AND MUST MODEL THE
RESPONSIBLE USE OF ARTIFICIAL
INTELLIGENCE AND IN THE PROCESS,
IT SHOULD PRIORITIZE THE SAFETY
AND WELL BEING OF IMPACTED
COMMUNITIES, ESPECIALLY LOW
INCOME COMMUNITIES AND
COMMUNITIES OF COLOR AS WELL AS
PUBLIC SECTOR WORKERS. MY
PERSPECTIVE COMES FROM SEVERAL
DECADES OF CONDUCTING INTENSIVE
RESEARCH ON THE U S¼ LABOR
MARKET AND THE FUTURE OF WORK.
FOR THE PAST FIVE YEARS, MY TEAM
AND I HAVE BEEN STUDYING DIGITAL
TECHNOLOGIES. A I AND THEIR
IMPACT ON FRONT LINE WORKERS
WITH THE GOAL OF ENSURING THAT
WORKING FAMILIES IN CALIFORNIA
HAVE THE TOOLS THEY NEED TO
THRIVE IN THE 21ST CENTURY
ECONOMY. IT WON'T SURPRISE YOU
TO HEAR THAT OUR RESEARCH HAS
DOCUMENTED THAT BOTH PRIVATE AND
PUBLIC SECTOR EMPLOYERS IN THE U
S¼ AND IN CALIFORNIA ARE
INCREASINGLY USING DIGITAL
TECHNOLOGIES IN THE WORKPLACE.
THEY ARE CAPTURING, BUYING AND
ANALYZING WORKER DATA. THEY ARE
ELECTRONICALLY MONITORING THEIR
WORKERS THEY ARE USING.
ALGORITHMIC MANAGEMENT TO
MANAGEMENT MANAGE THEM, AND
THEY'RE USING TECHNOLOGY TO
AUGMENT AND AUGMENT AUTOMATE
WORKERS TASKS. I WANT TO
EMPHASIZE THE UNDERLYING
TECHNOLOGIES RANGE ALL THE WAY
FROM VERY SIMPLE DATA ANALYTICS
TO GENERATIVE A. I AND I WOULD
SIMILARLY REALLY URGE THE
SENATORS TO TAKE A BROAD LENS
HERE ON THE TYPES OF
TECHNOLOGIES WE ARE TALKING
ABOUT. WE ARE OFTEN TALKING
ABOUT INTEGRATED SYSTEMS THAT
DON'T FEATURE JUST ONE TYPE OF
TECHNOLOGY. AND I UNDERSTAND THE
GENERATIVE A. I POSES ITS OWN.
CONCERNS AND RISKS. BUT WE HAVE
TO UNDERSTAND THAT IN PRACTICE,
WE ARE GOING TO BE TALKING ABOUT
ENTIRE RANGE OF TECHNOLOGIES.
FOR US. THE IMPORTANT POINT IS
THAT EMPLOYERS ARE USING THESE
TECHNOLOGIES TO MAKE HIGHLY
CONSEQUENTIAL EMPLOYMENT RELATED
DECISIONS, AND WE ARE ALREADY
SEEING CONCERNS EMERGE. INTENSE
MONITORING CAN PUSH WAREHOUSE
WORKERS TO THE POINT OF INJURY.
BIASED HIRING ALGORITHMS CAN
SHUT WOMEN AND WORKERS OF COLOR
OUT OF OPPORTUNITY. GIG PLATFORM
WORKERS CAN END UP MAKING BELOW
THE MINIMUM WAGE. SOME EMPLOYERS
ARE USING SURVEILLANCE TO
IDENTIFY WORKERS WHO ARE
EXERCISING THEIR RIGHT TO
ORGANIZE A UNION AND JUST IN
TIME SCHEDULING IN THE RETAIL
INDUSTRY CAN WREAK HAVOC ON
WORKERS' LIVES. I WANT TO
EMPHASIZE THAT SUCH HARMS ARE
NOT INEVITABLE. I BELIEVE THAT
EMPLOYERS CAN USE DATA DRIVEN
TECHNOLOGIES IN WAYS THAT
BENEFIT BOTH WORKERS AND THEIR
EMPLOYERS, AND WE HAVE PLENTY OF
EXAMPLES LIKE THAT, BUT IT WILL
TAKE RO ROBUST GUARD RAILS TO
ENSURE THAT WORKERS ARE NOT
HARMED BY A RAPIDLY SET OF
TECHNOLOGIES THAT ARE OFTEN
UNPROVEN, UNTESTED AND I WILL
SAY ARE OFTEN NOT AT ALL
UNDERSTOOD BY THE EMPLOYERS THAT
ARE USING THEM OR EVEN SOME OF
THE ENGINEERS THAT ARE
DEVELOPING THEM. SO THE NEED FOR
OVERSIGHT IS ESPECIALLY
IMPORTANT IN THE PUBLIC SECTOR.
WE ARGUABLY THE STAKES ARE
HIGHER AS ALL OF YOU HAVE
FLAGGED AND I WANT TO EMPHASIZE,
THOUGH, THAT THE STAKES ARE
HIGHER, PARTLY BECAUSE THE
TECHNOLOGY IS DUAL FACING IT
WILL IMPACT THE PUBLIC AND
WORKERS AND ON THE PUBLIC SIDE.
AS OTHERS HAVE SAID THE STATE'S
USE OF A I CAN HAVE PROFOUND
ECONOMIC SOCIAL INEQUITY IMPACTS
HERE I AM THINKING ABOUT
FUNCTIONS SUCH AS ELIGIBILITY
DETERMINATION FOR PUBLIC
BENEFITS LIKE U, I AND ME RISK
SCORING IN HUMAN SERVICES
PROGRAMS, GRADING AND CURRICULUM
DEVELOPMENT IN DELIVERY IN
COMMUNITY COLLEGES AND
SENTENCING IN THE CRIMINAL
JUSTICE SYSTEM, AND SIMILARLY A
I HAS A POTENTIAL TO
FUNDAMENTALLY TRANSFORM PUBLIC
SECTOR WORK AS WELL. IDEALLY. AS
OTHERS HAVE SAID IT COULD
AUTOMATE ROUTINE FUNCTIONS AND
FREE UP WORK TO WORKERS TO FOCUS
ON THEIR CLIENTS AND MAKE HIGHER
HIGHER ORDER DECISIONS AND
TROUBLESHOOT, BUT RESEARCHERS
ARE ALREADY DISCOVERING CONCERNS
AMONG THE PUBLIC SECTOR
WORKFORCE. THEY INCLUDE MENTAL
HEALTH STRESS FROM INCREASED
WORKLOADS JOB DESKILLING
BLURRING OF BOUNDARIES BETWEEN
HOME AND WORK, INADEQUATE
TRAINING SUPPORTS AND CRUCIALLY
HAVING TO SCRAMBLE AND BEING
BLAMED WHEN NEW TECHNOLOGY
ARRIVES AND DOESN'T WORK. SO HOW
SHOULD CALIFORNIA RESPOND? IN A
RECENT REPORT, WE IDENTIFIED
FOUR PRINCIPLES TO GUIDE POLICY
MAKERS. FIRST PUBLIC SECTOR
WORKERS SHOULD PARTICIPATE FULLY
IN DECISION MAKING ABOUT NEW
TECHNOLOGIES BECAUSE THEY
POSSESS POSSESS THE KNOWLEDGE
AND EXPERIENCE TO SUPPORT
RESPONSIBLE AND EFFECTIVE USE OF
A I IN DELIVERING PUBLIC
SERVICES. WE CALL THIS THE
WORKER IN THE LOOP PRINCIPLE,
MUCH LIKE THE MORE GENERAL HUMAN
IN THE LOOP, AND THAT IS THAT
PUBLIC SECTOR WORKERS CAN PLAY
CAN AND SHOULD PLAY A CRITICAL
ROLE IN SAFEGUARDING THE PUBLIC.
HARM WHEN GOVERNMENT IS USING
TECHNOLOGIES THAT ARE CAPABLE OF
AUTONOMOUS REASONING AND
DECISION MAKING BECAUSE THEY ARE
ON THE FRONT LINES. THESE
WORKERS ARE BEST POSITIONED TO
ANTICIPATE THE IMPACTS OF A I ON
EQUALITY, ACCESSIBILITY AND
EQUITY OF PUBLIC SERVICES, AND I
WOULD JUST STRESS LIKE OTHERS.
THE IMPORTANCE OF HUMAN REVIEW
AND OVERSIGHT IS CENTRAL TO MANY
LEADING POLICY FRAMEWORKS,
INCLUDING THE WHITE HOUSE'S
BLUEPRINT OF A I BLUEPRINT FOR A
I, A BILL OF RIGHTS AND THE
SEPTEMBER 2020 THREE REPORT BY
THE CALIFORNIA GOVERNMENT
OPERATIONS AGENCY. SECOND AS THE
SENATORS HAVE NOTICED, WE MUST
INVEST IN PUBLIC SECTOR WORKERS
TO ENSURE THEY HAVE THE SKILLS
AND EXPERTISE TO SELECT,
ADMINISTER, MANAGE AND WORK
ALONGSIDE PUBLIC SECTOR A I. THE
PRIORITY SHOULD BE TO BUILD THIS
EXPERTISE IN HOUSE GIVEN THE
IMPORTANCE OF CENTERING THE
PUBLIC GOOD IN THE CONTEXT WHERE
PRIVATE SECTOR INTERESTS
SOMETIMES DOMINATE. IN FACT, THE
OFFICE OF MANAGEMENT AND BUDGET
RECENTLY CALLED FOR A FULLY
TRAINED PUBLIC SECTOR WORKFORCE
IN ITS MEMORANDUM TO THE HEADS
OF FEDERAL AGENCIES WHO ARE
IMPLEMENTING PRESIDENT BIDEN'S
EXECUTIVE ORDER ON A I AND HERE
I WOULD FLAG THAT SOME OF THE
MOST SUCCESSFUL MODELS FOR
INCUMBENT WORKER TRAINING IN THE
U S¼ HAVE BEEN LABOR MANAGEMENT
TRAINING PARTNERSHIPS IN THESE
PARTNERSHIPS, UNIONS AND HIGH
ROAD EMPLOYERS COLLABORATE TO
SOLVE KEY CHALLENGES IN THEIR
INDUSTRY WHILE ENSURING THE
ECONOMIC SECURITY OF THEIR
WORKERS. I RECOMMEND DRAWING ON
CALIFORNIA'S STRONG TRACK RECORD
HERE IN SUPPORTING THESE TYPES
OF PARTNERSHIPS THROUGH ITS
FUNDING OF THE HÌ R Tð P
PROGRAMS, WHICH I CAN TELL THE
SENATORS ABOUT MORE. THIRD
PUBLIC SECTOR WORKERS SHOULD
HAVE THE RIGHT TO ORGANIZE AND
BARGAIN OVER THE STATE'S
PROCUREMENT AND DEPLOYMENT OF
NEW TECHNOLOGIES AS A MANDATORY
SUBJECT OF BARGAINING, BOTH THE
GOVERNOR'S EXECUTIVE ORDER AND
THE SENATORS BILLS INCLUDE
PROVISIONS FOR THE INVOLVEMENT
OF WORKER REPRESENTATIVES, WHICH
IS A GREAT START. I WOULD JUST
REALLY STRESS IT IN PRACTICE. IT
IS VITALLY IMPORTANT THAT
WORKERS IN THEIR UNIONS ARE
BROUGHT IN FROM THE BEGINNING AT
THE POINT OF PROBLEM DEFINITION
BEFORE THE PROCUREMENT PROCESS.
ARTS NOT AFTER THE FACT WHEN
CRITICAL DECISIONS HAVE BEEN
MADE. AND THEN FINALLY,
CALIFORNIA SHOULD ADOPT MORE
ROBUST STANDARDS OF PROTECT
WORKERS. PUBLIC SECTOR WORKERS
AGAINST POTENTIAL HARMS FROM A I
BASED SYSTEMS, MUCH LIKE THE
PUBLIC. IN A RECENT REPORT FROM
MY PROGRAM, WE LAY OUT A FULL
SWEEP OF SUCH STANDARDS,
INCLUDING TRANSPARENCY AND
DISCLOSURE WORKER RIGHTS OVER
THEIR DATA AND ROBOTS, GUARD
RAILS ON THE STATE IN ITS ROLE
AS AN EMPLOYER. PREVENT HARMS
FROM ELECTRONIC MONITORING AND
ALGORITHMIC MANAGEMENT.
IMPORTANTLY I WOULD FLAG FOR THE
SENATORS THAT WORKERS CALIFORNIA
WORKERS AT LARGE PRIVATE SECTOR
EMPLOYERS ALREADY HAVE SOME OF
THESE RIGHTS AFTER THEY GAIN
COVERAGE UNDER THE CALIFORNIA
CONSUMER PRIVACY ACT ON JANUARY
1 OF LAST YEAR. SO IN CLOSING
THE GOVERNOR'S EXECUTIVE ORDER,
AND THE SENATOR'S BILLS ARE
REALLY ONLY THE FIRST STEP IN
WHAT WILL BE MONTHS AND YEARS OF
WORK BY THE STATE OF CALIFORNIA
TO ENSURE THE RESPONSIBLE
INTRODUCTION OF A I IN
GOVERNMENT SERVICES, AND I JUST
WANT TO UNDERLINE AGAIN THE
GREAT OPPORTUNITY THAT WE HAVE
AS A STATE TO MAKE SURE THE
PUBLIC SECTOR WORKERS ARE
INCLUDED AS CRITICAL STATE
STAKEHOLDERS TO MAKE SURE THE
PUBLIC BENEFITS FROM AND IS NOT
HARMED BY PUBLIC SECTOR. A. I
THANK YOU SO MUCH FOR
OPPORTUNITY, AND I'M HAPPY TO
TAKE ANY QUESTIONS. THANK YOU.
THANK YOU VERY MUCH. UH,
DIRECTOR. MY QUESTION TO YOU WAS
ANSWERED IN TESTIMONY. SO THANK
YOU FOR THAT BRIEFLY TO MS COOK.
AS YOU KNOW, THERE'S A LOT OF
VAST AMOUNTS OF TRAINING DATA
REQUIRED TO INFORM ALGORITHMS
OFTEN I'M JUST CURIOUS ABOUT
YOUR PRACTICE, UH OR YOUR
OBSERVATIONS IN THE PRIVATE
SECTOR APPROACH TO SAFEGUARDING
PRIVACY. UH, GIVEN THAT FACT IF
YOU COULD COMMENT ON THAT SO WE
ARE WE TALKING ABOUT THE TENSION
BETWEEN PRIVACY AND FAIRNESS.
PERHAPS I'M TALKING ABOUT THE
PRIVATE SECTOR, GIVEN HOW YOU
ARE SITUATED, NOT NECESSARILY
COMPANY SPECIFIC BUT YOUR
OBSERVATIONS IN THE SECTOR AND
IN THE PRACTICE HOW ARE WE,
GIVEN THE VAST AMOUNTS OF DATA
THAT IS NEEDED ON TRAINING, UH,
TO HAVE AN ADEQUATE, UH,
BASICALLY MODEL, UH, TO INFORM
THE APPLICATIONS AND ALGORITHMS
IN THIS CASE, WHAT IS THE
APPROACH IN THE PRIVATE SECTOR
OR WHAT SHOULD BE THE APPROACH
TO SAFEGUARDING INTEGRITY OF
DATA? ABSOLUTELY. AND THIS IS
OBVIOUSLY SOMETHING WE THINK.
ABOUT ESPECIALLY IN THE EU WHERE
THERE IS SUCH A PROMINENT
PRIVACY LAW AND ALSO THEIR
ONBOARDING. A NEW A. I LAW SO
THAT, UM, YOU KNOW, NAVIGATING
THE REGULATORY BOUNDS OF EACH
JURISDICTION IS EXTREMELY
IMPORTANT FOR EVERY EVERY
TECHNOLOGY COMPANY, NOT JUST
GOOGLE. UM BUT I WOULD I WOULD
ANSWER IN THIS WAY, ESPECIALLY
IN THE STATE IN THE CONTEXT OF
PUBLIC SECTOR. THE UNBOUND
CHATBOTS THAT WE ARE VERY
FAMILIAR WITH AS OF LAST YEAR
ARE NOT GOING TO ALWAYS BE THE
RIGHT APPLICATION FOR PUBLIC
SECTOR AND I WOULD ACTUALLY
VENTURE TO SAY IT'S OFTEN NOT
THE RIGHT APPLICATION FOR PUBLIC
SECTOR. THE EXAMPLE I GAVE IN MY
TESTIMONY WAS ABOUT A VERY
BOUNDED MODEL, A MODEL THAT WAS
BOUND USING VERY SPECIFIC, UM,
CONTEXT, SO A CALL CENTER AND IT
WAS TRAINED WITH USING IN
PARTNERSHIP WITH THE NEW YORK
DEPARTMENT OF LABOR LABOR DATA
TO MAKE SURE WE'RE PROVIDING
OPTIMAL OUTCOMES AND MAKE SURE
THAT WE'RE SAFEGUARDING THE OF
THE CITIZENS OF NEW YORK USING
THE SYSTEM, SO I WOULD SAY IT'S
VERY CONTEXT SPECIFIC AND VERY
DEPENDENT ON THE APPLICATION.
THERE ARE OTHER APPLICATIONS
THAT I THINK ARE ALSO WELL
SUITED FOR A PUBLIC SECTOR
FUNCTION LIKE OUR ENTERPRISE
SEARCH, UM, TOOL, WHICH IS
ANOTHER BOUND PRODUCT. SO IF YOU
WORK FOR THE DEPARTMENT OF
EDUCATION, PERHAPS IN THE STATE
OF CALIFORNIA, AND YOU WANT TO
PARTNER WITH THE DEPARTMENT OF
HUMAN SERVICES, YOU CAN USE DATA
FROM THOSE AGENCIES TO CREATE A
BOUND SYSTEM WHERE YOU HAVE
PERHAPS A SEARCH FUNCTION THAT
ONLY LOOKS AT THE DATA IN FRONT
OF IT THAT YOU'RE POINTING TO.
IT DOESN'T REACH OUT TO THE
WORLD'S INTERNET DATA, WHICH
MIGHT CAUSE YOU KNOW, MIXED
RESULTS LIKE HALLUCINATIONS.
THERE ARE A LOT OF DIFFERENT
PRODUCTS OUT THERE AND SO I
THINK WHAT WOULD BE HELPFUL FOR?
UM POTENTIALLY THIS COMMITTEE IS
TO LOOK AT THE APPLICATIONS THAT
ARE VERY SPECIFIC TO DELIVERY OF
PUBLIC SERVICE. AND MAYBE BETTER
IDEA OF HOW SOMETIMES THOSE CHAT
BOTS, UM, THAT WERE USING, UM,
IN THE PUBLIC SORT OF FOR FUN TO
MEAL PLAN TO PLAN SOME VACATIONS
MAY NOT BE THE BEST FIT FOR OUR
GOVERNMENT USE AND CERTAINLY WE
DON'T WANT GOVERNMENT EMPLOYEES
USING THOSE SYSTEMS BECAUSE IT
WILL NOT PROTECT THE PRIVACY.
YOU WANT TO USE ENTERPRISE GRADE
PRODUCTS. THANK YOU, MS. COOK,
UH, CHAIRMAN DODD. SO YOU USE
THE WORD BOUND ABOUT FIVE TIMES,
I. I DON'T KNOW WHAT THAT MEANS
OR WHAT? THE REFERENCE IS THERE?
YEAH SO BOUND SO UNBOUND WOULD
BE THESE PAPERS RIGHT HERE. SO
UM, IF I'M A CHAT, BOT, I'M
ROAMING THE INTERNET. I'M
GOBBLING UP EVERY PIECE OF PAPER
I CAN FIND, WHEREAS FOR A
PRODUCT THAT I WOULD USE IN IN
MY EXAMPLE WITH THE DEPARTMENT
OF, UM UH UH, EDUCATION IN THE
DEPARTMENT OF HEALTH AND HUMAN
SERVICE FOR CALIFORNIA, I WOULD
SAY OK, I AM THE DEPARTMENT OF
HEALTH AND HUMAN. I AM GOING TO
USE THIS DATA SET, WHICH I'M
VERY FAMILIAR WITH WHICH I'VE
CLEANED AND I'M GONNA PARTNER
WITH MY FRIEND OVER HERE IN THE
DEPARTMENT OF EDUCATION. I'M
GONNA USE THIS DATA SET AND THEN
I'M GONNA POINT THE PRODUCT JUST
AT THOSE TWO PILES OF DOCUMENTS
TO TEST IT. I'LL SAY WHAT'S
TAYLOR SWIFT DOING NEXT WEEKEND,
AND IT WON'T KNOW BECAUSE IT'S
ONLY LOOKING AT THE DEPARTMENT
OF LABOR AND THE DEPARTMENT OF
HEALTH AND HUMAN SERVICES DATA
TO LOOK FOR INFORMATION AND
PATTERNS THAT WE'RE TRYING TO
FIND ACROSS BOTH DATA SETS
LIKELY TO FIND A MORE CREDIBLE
SOURCE WHEN YOU YES, SIR. THANK
YOU FOR THAT. I APPRECIATE THAT.
THANK YOU. ARE THERE OTHER
MEMBER QUESTIONS FOR THIS PANEL?
ALL RIGHT. WE THANK YOU BOTH FOR
YOUR TESTIMONY AND
PARTICIPATION, HAPPY TO
RECOGNIZE AND INTRODUCE OUR
THIRD PANEL, WHICH WILL LOOK AT
THE STATE ENTITY PERSPECTIVE.
FIRST. AMY TONG, SECRETARY OF
GOVERNMENT OPS. LEANNA BAILEY
CRIMMINS, WHO'S DIRECTOR
CALIFORNIA, Dð, Ó Tð, ANGELA
SHELL CHIEF PROCUREMENT OFFICER,
CALIFORNIA DEPARTMENT OF GENERAL
SERVICES, AND MONICA ERICKSON,
CHIEF DEPUTY DIRECTOR,
CALIFORNIA DEPARTMENT OF HUMAN
SERVICES. WELCOME MADAM
SECRETARY. WE WILL START BY
RECOGNIZING YOU THEN FOLLOW
FOLLOWING IN THE ORDER INDICATED
IN THE AGENDA AND PLEASE PROCEED
WHEN READY. WELCOME, AND THANK
YOU. GOOD MORNING CHAIR DOT AND
CHAIR PAT AND, UH SENATORS, AM I
NOT ON? ARE YOU ON NOW? OK
YOU'RE ON. OK I TURN IT ON OR
THE BACK ON. ALL RIGHT. I'LL
START IT AGAIN. TESTING THE
TECHNOLOGY. GOOD MORNING. YOUR
VIDEO AND YOUR THOUGHTS AND
MEMBERS OF THE COMMITTEE. THANK
YOU FOR THE OPPORTUNITY TO BE
HERE TODAY. UM, I AM AMY TOM,
SECRETARY OF THE GOVERNMENT
OPERATIONS AGENCY, AND, UH, IT
IS MY PLEASURE TO BE HERE LATER
ON INTRODUCING A STEAM PANEL OF
OUR STATE ENTITIES WHO IS GOING
TO BE BRIEFING YOU ALL ON, UH
WHAT THE STATE'S IMPLEMENTATION
OF JANITOR A. I GAVE UP UNDER
THE GOVERNOR'S EXECUTIVE ORDER
IS DELETING AND COORDINATING, UH
, AGENCY FOR THE IMPLEMENTATION
EFFORT IN PARTNERSHIP WITH THE
DEPARTMENT OF TECHNOLOGY
DEPARTMENT OF HÌ R DEPARTMENT OF
GENERAL SERVICES, THE OFFICE OF
A DATA INNOVATION, THE STATE
CYBERSECURITY INTEGRATION CENTER
GO BIZ AND THE STATE LABOR AND
WORKFORCE DEVELOPMENT AGENCY.
WHILE IMPLEMENTING THIS
EXECUTIVE ORDER. ONE OF OUR
PRIMARY INITIAL GOAL HAS BEEN
THE STUDY THAT DEVELOPMENT USE
AND THE RISK OF GENITIVE A I
THROUGH THE STATE. RECOGNIZING
THE IMPORTANCE OF EVALUATING
THIS TECHNOLOGY FROM THE START,
OUR FIRST DELIVERABLE WAS A
REPORT RELEASED IN NOVEMBER ON
THE BENEFITS AT RISK OF GENITIVE
A. I. THIS REPORT EXAMINES HOW
THE STATE COULD IMPROVE
OPERATIONS USING GEN. A. I AND
HIGHLIGHTS THE IMPORTANCE OF
SECURE CONTROL PILOTS IN DOING
SO. THE REPORT ALSO DESCRIBES
POTENTIAL RISK, NOTING THAT
THESE ISSUES MUST BE ADDRESSED
TO ENSURE THE CALIFORNIANS WERE
COLLECTIVELY BENEFIT FROM THIS
TECHNOLOGY. WHILE EVALUATING
GENAI WERE ALSO DEVELOPING A
DELIBERATE AND RESPONSIBLE
PROCESS FOR THE DEPLOYMENT OF
THIS TECHNOLOGY WITHIN THE STATE
GOVERNMENT BY EVALUATING THE
FEASIBILITY OF THESE PILOTS. FOR
THIS REASON WE ARE CARRYING OUT
MULTIPLE LEVEL OF ENGAGEMENT
WITHIN THE STATE, INCLUDING THE
STATE SUBJECT MATTER. EXPERTS
OUR LEGAL TEAMS ARE PROGRAM
TEAMS, OUR PRIVACY TEAMS,
SECURITY TEAMS PARTICIPATING IN
THE COMMUNITY OF A I PRACTICE AS
WELL AS THE GENITIVE A. I LEADS
FROM ONE IN 100 DEPARTMENTS. IN
ADDITION TO THE STATE TEAM, WE
CONSULTED EXTERNAL ENTITIES WITH
SPEAKING OVER 70 SEPARATE
ENTITIES WITH OVER 140 UNIQUE
INDIVIDUALS FOR THE ACADEMIA
CENTERS, MANY OF YOU HEARD FROM
EARLIER, LOCAL AND FEDERAL
OFFICIALS, COMMUNITY BASED
ORGANIZATIONS AS WELL AS THE
STATE LABOR LEADERS TO MAKE SURE
THAT WE KEEP OUR WORKFORCE AND
THE COMMUNITIES WE SERVE AT THE
FOREFRONT OF OUR EVALUATING THE
SOLUTION. OUR GOAL IS TO LEARN
BY DOING WE ARE TAKING THIS
RESPONSIBLE VERY, VERY
SERIOUSLY. AND THROUGH THESE
PREDE DEVELOPPED PILOTS THAT
WHICH YOU HAVE HEARD, AND I
BELIEVE MY COLLEAGUE'S GONNA GO
INTO A LITTLE BIT MORE. WE ARE
USING THE PUBLIC SECTOR DATA AND
ENGAGING OUR EMPLOYEES FROM THE
VERY BEGINNING IN DEVELOPING
THIS PROCUREMENT. AND WE ARE
WORKING TO MAKE SURE THAT WE'RE
NOT ONLY FOCUSING ON DOING THIS
WORK IN A SAFE, SECURE AND
PRIVACY PROTECTED MANNER WERE
ALSO ENGAGING, ADDITIONAL
COMMUNITY BASED ORGANIZATION AND
PARTNERS TO BE PART OF THE
SOLUTION EVALUATION AS THESE
PILOTS ARE MOVING FORWARD, AND
TO FURTHER DETERMINE WHETHER
THESE PILOTS WILL CONTINUE OR
NOT INTO A REAL SOLUTION IN
DELIVERING STATE SERVICES. IN
ADDITION, GIVEN HOW FAST GEN IS
EVOLVING, WE ARE COMMITTED TO
DEVELOPED OUR ADMINISTRATIVE
POLICY PROCESSES AND PROCEDURES
IN AN ITERATIVE MANNER AND TO
UPGRADE THEM OR UPDATE THEM ON
THE FREQUENCY THAT IT COULD BE
REFLECTING THE LEARNINGS OF OUR
PILOTS. THAT CONCLUDES MY
COMMENTS AND THANK YOU AGAIN FOR
YOUR TIME AND HAVING ME TO JOIN
THIS AND WE WELCOME THE
CONTINUED OVERSIGHT AND
COLLABORATION FROM LEGISLATIVE
MEMBERS AND AS WE GO DOWN TO
THIS JOURNEY TOGETHER, WITH THAT
I'M TURNING OVER TO DIRECTOR
LEANNA BAILEY KREMEN. THANK YOU,
SECRETARY. GOOD MORNING, MR
CHAIRS AND SENATORS. THANK YOU
FOR THE OPPORTUNITY TO SPEAK
WITH YOU TODAY. I AM LEANNA
BAILEY CRIMMINS, CALIFORNIA
STATE CHIEF INFORMATION OFFICER
AND THE DIRECTOR OF THE
CALIFORNIA DEPARTMENT OF
TECHNOLOGY CO PRESENTING WITH ME
TODAY IS ANGELA SHELL, WHO IS
THE STATE CHIEF PROCUREMENT
OFFICER FOR THE DEPARTMENT OF
GENERAL SERVICES. AND ALSO
MONICA ERICKSON, WHO'S THE CHIEF
DEPUTY OF KAI RAR. IN
PARTNERSHIP WITH GOV OPS. OUR
DEPARTMENTS PLAY A PIVOTAL ROLE
IN DELIVERING ON THE GOVERNOR'S
EXECUTIVE ORDER. GOVERNOR NEWSOM
IS COMMITTED TO CONTINUING
CALIFORNIA'S LEADERSHIP IN
EMERGING TECHNOLOGY, AND THIS
EXECUTIVE ORDER IS AN EXAMPLE OF
THAT COMMITMENT. OUR GOAL IS TO
CAPTURE THE BENEFITS OF
GENERATIVE A I FOR THE GOOD OF
SOCIETY, BUT ALSO TO BE
COMPLETELY AWARE OF THE
POTENTIAL HARM. AS MOST OF YOU
ARE AWARE, THE STATE OF
CALIFORNIA HAS EMBRACED
TECHNOLOGY AND HAS USED
ARTIFICIAL INTELLIGENCE SUCH AS
CAL FIRE AND DETECTING, UM FIRES
AND SAVING LIVES AND ALSO SAVING
OUR ENVIRONMENTAL RESOURCES, AND
WE'RE ALSO USING CHATBOTS
PREDICTIVE ANALYTICS. BUT WE
ALSO UNDERSTAND THAT GVA IS
UNIQUE AND DIFFERENT IN ITS OWN
ITS OWN RIGHT. UM CALIFORNIA IS
HOME TO 35 OF THE 50 TO WORLD'S
TOP ARTIFICIAL INTELLIGENCE
COMPANIES. AND SO THAT'S WHY WE
BUILT AN INCLUSIVE PROCESS TO
BRING THESE BEST OF MINDS
TOGETHER TO HELP US AS WE ARE
DEVELOPING THE GUIDELINES. THE
EXECUTIVE ORDER ESTABLISHED THE
PARTNERSHIP BETWEEN UC BERKELEY
AND STANFORD. UM AND WE WANTED
TO MAKE SURE THAT WE WERE HAVING
A THOUGHTFUL DISCUSSION ON
REALLY UNDERSTANDING HOW WE CAN
BENEFIT IN GOVERNMENT SERVICES,
BUT ALSO MAKING SURE THAT WE ARE
STEPPING UP WHEN IT COMES TO
LEADERSHIP TO PROTECT AGAINST
THE RISKS AND OBVIOUSLY MAKING
SURE THAT WE ARE AWARE OF THAT
AS WE'RE DEVELOPING OUR
PROCESSES AND PRACTICES. IN
ADDITION TO ACADEMIA, GOV OPS,
THE OFFICE OF DATA INNOVATION
CALL Ó E S¼. èC Dð, Tð, Dð G,
S¼ Kò HÌ R AND MANY OTHER STATE
DEPARTMENTS ARE ENGAGING
NUMEROUS STAKEHOLDERS YOU HAD
HEARD FROM MADAM SECRETARY OVER
70 JUST IN THE GUIDELINES THAT
WE'VE PRODUCED SO FAR. IN
ADDITION TO THIS, WE ARE LOOKING
AT PROVIDING A COMPREHENSIVE SET
OF DELIVERABLES RELATED TO
BENEFIT RISKS, PROCUREMENT USES
AND TRAINING. EVALUATING UH, OUR
CRITICAL INFRASTRUCTURE AND
MAKING SURE WE'RE PROTECTING
AGAINST ANY POTENTIAL HARM.
UNDERSTANDING VULNERABLE
COMMUNITIES AND HOW WE CAN MAKE
SURE THAT WE'RE NOT IN
INTRODUCING OR AMPLIFYING ANY
BIASES. LOOKING AT IT. PROJECT
APPROVAL PROCESSES, PROCUREMENT
CONTRACTS. HOW DO WE APPROVE
THESE PROJECTS MOVING FORWARD?
PROOF OF CONCEPTS AND PILOT
PROJECTS, AND THE EXECUTIVE
ORDER SPECIFICALLY REQUESTED
THAT THE CALIFORNIA DEPARTMENT
OF TECHNOLOGY BUILT A SANDBOX,
WHICH WAS A SAFE AND, UM, AREA
TO BE ABLE TO TEST THIS
INNOVATIVE SET OF SERVICES AND
ALSO TO DO A INVENTORY IN
ALIGNMENT WITH ASSEMBLY BILL
302. AS OF TODAY, THE STATE HAS
ISSUED FIVE REQUESTS FOR
INNOVATIVE IDEAS TO SAFELY AND
EFFECTIVELY TEST THE
IMPLEMENTATION OF GEN. A AND
ANGELA SHELL WILL GO INTO MORE
DETAIL ON THAT RF I SQUARE
PROCESS THAT SHE IS OVERSEEING.
AS A STATE CHIEF INFORMATION
OFFICER AND ALSO THE CO-CHAIR OF
THE CYBER SECURITY AT CALIFORNIA
CYBER SECURITY INTEGRATION
CENTER, WHICH IS CALCI. WE ARE A
CO CHAIR THAT WITH DEPARTMENT
UH, DIRECTOR WARD FOR, UH Kò Ó
E S¼. I TAKE CYBER SECURITY AND
PRIVACY, UM, OF OUR DATA
SERIOUSLY, WE HAVE A MOTTO,
PEOPLE FIRST SECURITY ALWAYS AND
THE DEPARTMENT OF TECHNOLOGY
SANDBOX THAT WE ARE BUILT WILL
BE MONITORED 2524 BY SEVEN BY
THE SECURITY EXPERTS IN THE
STATE. UM WE'RE ALSO HAVE
ISOLATED IT FROM THE EXISTING
OPERATING SYSTEM OF THE STATE,
SO IT'S IN ITS OWN ISOLATED
FIREWALLED ENVIRONMENT. AND THE
FIVE JIJA I. PROOF OF CONCEPTS
WILL ONLY USE PUBLICLY AVAILABLE
DATA THAT IS. SHARED AMONGST OUR
WEBSITES AND AVAILABLE TO ALL
CALIFORNIANS. SO THAT MEANS THAT
NÓ PERSONAL, CONFIDENTIAL,
PROPRIETARY OR SENSITIVE
INFORMATION WILL BE USED IN
REGARDS TO THESE PROOF OF
CONCEPTS. BUT EVEN THOUGH WE'RE
USING PUBLICLY AVAILABLE DATA, I
JUST TOLD YOU WE'RE DOING
SECURITY ALWAYS. SO WE HAVE
BUILT HIGH WIDE. UM WALL IS VERY
HIGH AND, UM, MOATS VERY DEEP TO
MAKE SURE THAT ALL SAFEGUARDS
ARE IN PLACE. SO SOME OF THOSE
SAFEGUARDS INCLUDE FED RAMP
CERTIFICATION THAT IS THE
HIGHEST LEVEL OF FEDERAL
GOVERNMENT APPROVED REQUIREMENTS
. WHEN IT COMES TO CLOUD
SECURITY. WE ALSO HAVE THE
HIGHEST DATA, UM, LEVELS OF
ENCRYPTION BOTH IN REST AND IN
TRANSIT, AND WE HAVE REQUIRED
CONTROLS FOR ACCESS SO THAT ONLY
INDIVIDUALS THAT HAVE THE RIGHTS
CAN ACCESS THE INFORMATION. WE
ALSO HAVE SPECIFIC DATA
MANAGEMENT REQUIREMENTS, AS MOST
OF YOU KNOW, WITH OUR GENERAL
PROVISIONS, WHAT THE STATE
PRODUCES WHAT THE STATE OWNS AND
SO THE STATE WILL RETAIN
OWNERSHIP AND CONTROL OF ALL OF
ITS DATA, INTELLECTUAL PROPERTY
AND ONGOING VISIBILITY AND HOW
THAT STATE DATA IS BEING USED
AND SECURED. ALL PARTICIPATING
VENDORS MUST COMPLY WITH OUR
SECURITY POLICIES AND OUR CLOUD
SECURITY GUIDES, WHICH REQUIRES
VENDORS TO CLASSIFY. SEQUESTER
PROTECT STATE DATA AND
CONTINUOUSLY MONITOR THAT DATA
FOR ANY POTENTIAL WARNING SIGNS
OF A SECURITY OR PRIVACY
INCIDENT. I LOOK FORWARD TO
FURTHER FURTHER ENGAGING THE
LEGISLATURE ON THIS AND OTHER
EMERGING TECHNOLOGY INNOVATIONS,
AND THAT COMPLETES MY OPEN
REMARKS AVAILABLE FOR QUESTIONS.
THANK YOU, DIRECTOR. GOOD
MORNING, MR CHAIR AND SENATORS
AND THANK YOU FOR THE
OPPORTUNITY TO PRESENT TODAY
AGAIN. MY NAME IS ANGELA SHELL.
I AM THE STATE'S CHIEF
PROCUREMENT OFFICER AND ALSO THE
DEPUTY DIRECTOR FOR THE
PROCUREMENT DIVISION AT THE
DEPARTMENT OF GENERAL SERVICES.
MY ROLE IS TO OVERSEE THE
STATE'S PROCUREMENT OPERATIONS,
INCLUDING ISSUING PROCUREMENT
POLICY AND GUIDANCE FOR ALL OF
OUR STATE DEPARTMENTS IN
PARTNERSHIP WITH THE GOVERNMENT
OPERATIONS AGENCY AND èC Dð Tð
HAVE BEEN ACTIVELY WORKING TO
IDENTIFY WAYS IN WHICH OUR STATE
PROCUREMENT PROCESSES CAN BE
ENHANCED TO PROVIDE DEPARTMENTS
WITH THAT GUIDANCE NEEDED TO
EFFECTIVELY PROCURE GENERATIVE
A. I TOOLS WHILE ALSO PROVIDING
GUARD RAILS AROUND WHEN
GENERATIVE A. I IS BEING USED IN
OUR PROCUREMENT AND HOW TO
SAFEGUARD THAT USE THROUGH OUR
CONTRACTUAL PROTECTIONS THAT
INCLUDE THINGS LIKE NOTICES FROM
VENDORS WHEN MATERIALS ARE
GENERATED BY A GEN A. I AND THEN
ADDITIONAL TERMS AND EXCUSE ME
ADDITIONAL TERMS AND CONDITIONS
THAT CAN HELP TO PROTECT. UM,
WHATEVER WE'RE PURCHASING.
THAT'S THAT'S GENERATED BY GEN.
A I, UM AS THE DIRECTOR TOUCHED
ON THE STATE IS USING OUR MOST
INNOVATIVE PROCUREMENT PROCESS,
UM, KNOWN AS THE, UH, REQUEST
FOR INNOVATIVE IDEAS. TWO SO RF
I SQUARED. UM AND WHAT WE HOPE
TO GAIN OUT OF THIS IS AN
OPPORTUNITY FOR US TO HAVE SOME
INSIGHT, UH, USING THE SECURE
SANDBOX INTO HOW GENERATIVE A I
CAN BE USED BY THE STATE WITH
SOME SPECIFIC PROJECTS. SO TO
THAT END, THE RF I TWO PROCESS
ALLOWS THE STATE TO SOLICIT
INNOVATIVE IDEAS WITHIN AN
INDUSTRY BY RELEASING A PROBLEM
STATEMENT IDENTIFYING THE
CURRENT AS IS ENVIRONMENT AND
REQUESTING SOLUTIONS FROM THE
SUPPLIER COMMUNITY OR INNOVA
INNOVATORS AS WE CALL THEM,
RATHER THAN IDENTIFYING STATE
DEFINED REQUIREMENTS UP FRONT.
THE RF I PROCESS ALLOWS FOR AN
UPFRONT VALIDATION OF PROPOSED
SOLUTIONS THROUGH WORKING MODELS
OR WHAT WE'RE CALLING PROOFS OF
CONCEPT. SO THE STATE CAN MAKE
INFORMED DECISIONS ABOUT THAT
INNOVATIVE SOLUTION. IT ALLOWS
US TO ENTER INTO CONTRACTS WITH
A SINGLE OR MULTIPLE INNOVATORS
WITH THE ABILITY TO NEGOTIATE.
CONTRACT FOR FULL IMPLEMENTATION
ONCE THE PROOF OF CONCEPT PHASE
IS COMPLETE. AN RF TWO DOES
INVOLVE A COMPETITIVE BIDDING
PROCESS. IT ALSO ALLOWS AN
ITERATIVE DIALOGUE BETWEEN THE
STATE AND THOSE BIDDING VENDORS.
THE RF I TWO IS ADVERTISED ON
THE STATE'S GOVERNMENT. E
MARKETPLACE, CALI PROCURE AND
PROPOSALS ARE SUBMITTED AT A
DATE SPECIFIED IN THE ADVERTISED
DOCUMENT. A PROPOSAL CONFERENCE
IS HELD FOR INTERESTED VENDORS
TO ASK CLARIFYING QUESTIONS OF
THE STATE AROUND THE PROCUREMENT
PROCESS OR THE PROBLEM STATEMENT
ITSELF AND THOSE RESPONSES BY
THE STATE TO ANY QUESTIONS ARE
POSTED ONLINE FOR ALL POTENTIAL
BIDDERS TO SEE BEFORE THEY
SUBMIT A PROPOSAL FOR THE
DEADLINE. THE PROPOSAL OR
CONCEPT PAPERS ARE EVALUATED BY
A TEAM OF SUBJECT MATTER.
EXPERTS AND POTENTIAL VENDORS
ARE PLACED INTO A POOL FOR
EVALUATION OF THEIR IDEAS IN
THAT PROOF OF CONCEPT PHASE. THE
STATE. UH, NEGOTIATES CONTRACTS
WITH THOSE POTENTIAL VENDORS WHO
WILL TEST THOSE CONCEPTS AND
AGAIN. ADDITIONAL CONTRACTS MAY
BE ISSUED FOR A FULLY SCALED UP
SOLUTION ONCE THE PROOF OF
CONCEPT PHASE IS COMPLETE. THE
STATE IS USING THE R 52 FOR ALL
FIVE OF THE PROOFS OF CONCEPT,
UM AND THE FOLLOWING FIVE
PROJECTS ARE THERE ARE TWO WITH
THE CALIFORNIA DEPARTMENT OF
TRANSPORTATION. UH, ONE IS
CALLED VULNERABLE ROADWAY USER,
AND IT INVESTIGATES NEAR MISSES
OF INJURIES AND FATALITIES TO
IDENTIFY RISKY AREAS AND MONITOR
INTERVENTIONS THAT ARE DESIGNED
TO INCREASE SAFETY OF VULNERABLE
ROAD USERS. THE SECOND ONE IS
TRAFFIC MOBILITY, INSIGHTS, AND
THE GOAL OF THAT, UH, IS TO
PROCESS AND INTERPRET COMPLEX
DATA TO IMPROVE TRAFFIC PATTERN
ANALYSIS, ADDRESS BOTTLENECKS
AND ENHANCE OVERALL TRAFFIC
MANAGEMENT. THE THIRD IS WITH
THE CALIFORNIA DEPARTMENT OF TAX
AND FEE ADMINISTRATION TO CALL
CENTER TEAM PRODUCTIVITY. UM AND
THE GOAL OF THAT IS TO IMPROVE
CALL CENTER OPERATIONS TO REDUCE
CALL TIMES, IMPROVE CUSTOMER
SERVICE AND INCREASE TAXPAYER
COMPLIANCE. THE LAST TWO ARE
WITH THE CALIFORNIA HEALTH AND
HUMAN SERVICES AGENCY. ONE IS
FOR LANGUAGE ACCESS, AND THE
GOAL OF THAT IS TO ENSURE THAT
CALIFORNIANS WITH LIMITED
ENGLISH PROFICIENT PROFICIENCY
HAVE TIMELY ACCESS TO
INFORMATION ABOUT PUBLIC
BENEFITS, AND THEY CAN THE GATE
PUBLIC PROGRAMS WITH EASE AND
WITHOUT CONFUSION, AND THE LAST
IS A HEALTH CARE FACILITY
INSPECTIONS AND THE GOAL OF THAT
ONE IS TO LEVERAGE TOOLS TO
EXPEDITIOUSLY DOCUMENT THE FACTS
OR FINDINGS THAT ARE SEEN BY A
SURVEYOR DURING HEALTH CARE
FACILITY INSPECTIONS, AND
HOPEFULLY THE TOOL WILL HELP
THEM DEVELOP A CONCRETE SET OF
OUTCOMES OR CITATIONS THAT MATCH
THE STATE AND FEDERAL
REQUIREMENTS. THE TIMELINES FOR
THE R FA. UH UH, PROCESS ARE ALL
FIVE PROJECTS HAVE BEEN
ADVERTISED AND CONCEPT PAPERS
HAVE BEEN SUBMITTED TO THE
STATE. THE CONCEPT PAPERS ARE
CURRENTLY IN THE EVALUATION
PHASE OF THE PROCUREMENT PROCESS
. SUCCESSFUL VENDORS ARE
ANTICIPATED TO BEGIN TESTING CON
. EXCUSE ME CONCEPTS IN THE
SANDBOX IN MARCH OR APRIL, AND
THEN TESTING IS ANTICIPATED TO
BE COMPLETE WITHIN SIX MONTHS IN
THAT SANDBOX. AND THEN IF
APPLICABLE AGAIN, IMPLEMENTATION
CONTRACTS WILL BE ISSUED ONCE
THE TESTING AND ANALYSIS IN THE
SANDBOX IS COMPLETE, SO THAT
COMPLETES MY REMARKS, AND I'M
HAPPY TO ANSWER ANY QUESTIONS.
THERE WE GO. GOOD MORNING, UH
MONICA ERICKSON, CHIEF DEPUTY
DIRECTOR OF DEPARTMENT OF HUMAN
RESOURCES. THE EXECUTIVE ORDER
CALLS FOR TRAINING TO SUPPORT
STATE GOVERNMENT WORKFORCE TO
PREPARE AND SUPPORT STATE
GOVERNMENT WORKFORCE FOR
GENERATIVE. A I. FUTURE TRAINING
MAY INCLUDE EDUCATION AND
TRAINING. COLLABORATION AND
COMMUNICATION. UPSKILLING AND
RESKILLING. SUPPORT AND
FEEDBACK. AND CONTINUOUS
LEARNING BEFORE J TOOLS ARE
DEPLOYED. AS A FIRST STEP TO
PREPARE THE WORKFORCE FOR
GENERATIVE. A I DIALOGUE AND
ENGAGEMENT WILL GUIDE THE DESIGN
OF THE TRAININGS WITH THE GOAL
OF INDIVIDUALS GAINING GENERAL
EDUCATION NEEDED SO THAT
POTENTIAL LABOR LEGAL AND
PRIVACY RISKS ARE IDENTIFIED AND
ADDRESSED PROACTIVELY. ALSO
BUILDING GENAI SKILLS AND
COMPETENCIES TO IDENTIFY WHERE
GEN. A MAY HELP IMPROVE
OPERATIONAL EFFICIENCY, HIGH
QUALITY EQUITABLE SERVICE
DELIVERY. SECURITY AND SAFETY
ARE PARAMOUNT IN GENERATIVE. A.
I, AS MY COLLEAGUES HAVE STATED
TRAINING WILL BE DESIGNED TO
ENSURE SAFETY AND SECURITY
DURING USE CASES AND PILOTING.
SCALING THE NECESSARY
FAMILIARITY WITH THE WAY
GENERATIVE A I. TECHNOLOGY WORKS
WITHIN THE WORKFORCE WILL ENSURE
THAT ALL DEPARTMENTS ARE WELL
PREPARED. RESPOND TO ANY
QUESTIONS. STATE GOVERNMENT
EMPLOYEES MAY HAVE AND ARE READY
TO LEVERAGE THEIR IDEAS FOR
POSITIONING GENAI TO IMPROVE
CRITICAL SERVICES THEY DELIVER
FOR CALIFORNIA. THANK YOU SO
MUCH FOR ALLOWING ME TO PRESENT
TODAY AND THAT CONCLUDES MY
COMMENTS. THANK YOU AGAIN TO ALL
OF THE PANELISTS AND JUST
BRIEFLY FOR Dð Ó Tð WITH
RESPECT TO THE EVALUATIVE
PROCESS ON RF. I DID YOU HAVE
YOU INSTITUTIONALIZED OR CREATE
A PROTOCOL THAT THERE'S A
SPECIFIC INVOLVEMENT FOR PUBLIC
SECTOR EMPLOYEES IN BOTH THE
SUBMITTAL AND DEVELOPMENT OF
PROBLEM STATEMENTS AND THEN THE
PILOT EVALUATION? THANK YOU,
SENATOR PADILLA FOR THAT
QUESTION. YES THE STATE
WORKFORCE IS JUST LIKE ANY
EMERGING TECHNOLOGY IS VITAL
THROUGH THE ENTIRE LIFE CYCLE.
WORKERS DETERMINE THE BUSINESS
CHALLENGE AND GOALS. THEY ARE
ALSO SETTING UP WHAT IS REQUIRED
AS REQUIREMENTS AND OUTCOMES AND
MEASURABLE OUTCOMES. THEY ARE
ALSO INVOLVED IN EVALUATING WHAT
A VENDOR PROPOSES AND THEN BASED
ON THE SOLUTION. THE OUTPUTS
THAT COME OUT OF THAT IS ALSO
NEEDING A HUMAN IN THE LOOP. OR,
AS WE HEARD EARLIER, A WORKER IN
THE LOOP TO VALIDATE THAT
OUTCOME, SO THEIR INCREMENTALLY
INVOLVED ALL THE WAY FROM THE
BEGINNING TO THE END. THANK YOU,
DIRECTOR. OTHER QUESTIONS FOR
YOU, CHAIRMAN DODD. ALL RIGHT,
SIR, ARE THERE OTHER MEMBERS WHO
HAVE ASK THE SENATOR Ó NOT NOT
A QUESTION. MARK OR COMMENT,
COMMENT. SO I WAS JUST EXCITED
TO HEAR THE DIFFERENT AREAS THAT
YOU FOLKS ARE CURRENTLY WORKING
ON ANOTHER AREA THAT JUST TO
PLAN TO SEED FOR FUTURE
CONSIDERATION WOULD BE USING THE
PROGRAMS IN ORDER TO TRANSLATE
THE POLICY. THE LEGISLATION THAT
WE ARE CURRENTLY INTRODUCING,
AND SO THAT, UH, COMMUNITIES
WITHIN CALIFORNIA HAVE ACCESS TO
THE ACTUAL LEGISLATION THAT'S
BEING PROPOSED AND VETTED, AND I
THINK IT WOULD BE A GREAT
OPPORTUNITY TO REALLY ENGAGE ALL
CALIFORNIANS. UM IN THAT
DISCUSSION OF POLICY. JUST
THOUGHT IT WOULD BE GREAT. HEARD
YOU AND GOT ME REALLY EXCITED ON
THAT END, BECAUSE THAT'S
SOMETHING THAT IT'S BEEN, UH,
WORKING THROUGH MY THROUGH MY
HEAD AS FAR AS HOW DO WE MAKE
THIS MUCH MORE ACCESSIBLE TO ALL
CALIFORNIA? SO THANK YOU. THANK
YOU. THANK YOU, SENATOR AND
AGAIN JUST DOESN'T APPEAR THAT
THERE'S ADDITIONAL SORRY,
CHAIRMAN DODD. YEAH, I. I DON'T
HAVE ANY SPECIFIC QUESTIONS, BUT
I I'LL TELL YOU I AM VERY
IMPRESSED WITH THE FOUR OF YOU
AND WHAT WE'RE DOING HERE IN THE
STATE OF CALIFORNIA. UH, YOU
KNOW IN THIS AREA AND IT REALLY
APPRECIATE YOUR EFFORTS AND YOUR
LEADERSHIP, UH, I THINK IT GIVES
US A LOT OF TRUST IN WHAT'S
GOING ON, AND, UH, I JUST THANK
YOU APPRECIATE IT ECHO THAT.
THANK YOU, MR CHAIRMAN, AND WE
WILL TURN IT BACK TO CHAIRMAN
DODD FOR THE PUBLIC COMMENT
PORTION OF THIS PROCEEDING. AND
THANKS TO OUR DISTINGUISHED
PANELISTS, ALL THANK YOU, MADAM
SECRETARY. THANK YOU, DIRECTOR.
THANK YOU VERY MUCH. CHAIRMAN
PADILLA. NOW WE MOVE TO PUBLIC
COMMENT. THOSE IN THE AUDIENCE
THAT WISH TO UH COME UP. PLEASE
COME FORWARD NOW A FORMER LINE
AT THE MICROPHONE AND WE'LL
PROCEED. SO GOOD MORNING.
KNOWING WHAT TO BE FIRST. I'LL
TAKE THAT ONE, THEN. GOOD
MORNING, ESTEEMED COMMITTEE
MEMBERS. MY NAME IS PETER LAROE
MUNOZ. I'M WITH THE SILICON
VALLEY LEADERSHIP GROUP.
ARTIFICIAL INTELLIGENCE HAS THE
POTENTIAL TO HELP SOCIETY AND
THE GOVERNMENT UNDERSTAND AND
DEVELOP SOLUTIONS TO OUR
GREATEST CHALLENGES WHETHER
CLIMATE CHANGE, PUBLIC SAFETY,
TRANSPORTATION EFFICIENCY AND
OTHERS, CALIFORNIA HAS
ESTABLISHED ITSELF AS THE GLOBAL
LEADER OF A I RESEARCH, FUNDING
AND INNOVATION AND AS SUCH,
GOLDEN STATE POLICIES WILL SHAPE
A I'S TRA. IN THE STATE AND
BEYOND. THEREFORE THE SILICON
VALLEY LEADERSHIP GROUP HAS
ANNOUNCED THE FORMATION OF THE
INSTITUTE FOR CALIFORNIA
ARTIFICIAL INTELLIGENCE POLICY,
KNOWN AS CAP'S MISSION IS TO
WORK WITH ELECTED OFFICIALS AND
REGULATORS TO THINK THROUGH THE
OPPORTUNITIES AND CHALLENGES
PRESENTED BY A I AND IDENTIFY
AND PROMOTE POLICY AND
REGULATORY SOLUTIONS THAT ARE
INFORMED, BALANCED, RESPONSIBLE
AND EFFECTIVE. S¼ WORK IS
GROUNDED ON SEVERAL KEY
PRINCIPLES. WE BELIEVE A I
POLICY SHOULD FOCUS ON OUTCOMES,
NOT THEORETICAL, OR UN
DEMONSTRATED RISKS. FURTHER
POLICY SHOULD BE TECH AGNOSTIC
AND RECOGNIZE THAT THERE IS NÓ
ONE SIZE FITS ALL APPROACH OR
SYSTEM FOR ALL CHALLENGES.
FINALLY GOVERNMENT AND INDUSTRY
SHOULD PROMOTE RESPONSIBLE. A I
THAT IS HUMAN CENTERED. HAS
RESULTS THAT ARE APPROPRIATE FOR
THE USER AVOIDS PERPETUATING OR
REINFORCING BIASES SECURE FROM
UNAUTHORIZED ACCESS OR USE AND
CONSIDERATE OF THE BALANCE
BETWEEN BENEFITS AND CHALLENGES
OF DIFFERENT USE CASES. THE
SILICON VALLEY LEADERSHIP GROUP
AND THE INSTITUTE FOR CALIFORNIA
A I POLICY LOOK FORWARD TO
SERVING AS A RESOURCE FOR
POLICYMAKERS AND REGULATORS AS
WE LOOK TO MAXIMIZE THE BENEFITS
OF THIS TECHNOLOGY AND ADDRESS
COMPLEXITIES FOR ALL
CALIFORNIANS. THANK YOU.
SPEAKER. GOOD MORNING. GOOD
MORNING. UH, SANDRA BARRERA ON
BEHALF OF S¼ EIU, CALIFORNIA. UM
I WANT TO EMPHASIZE THE
IMPORTANCE OF CONSULTING RANK
AND FILE MEMBERS WHO ARE EXPERTS
ON HOW TO IMPROVE A SPECIFIC JOB
FUNCTION BEFORE PROCUREMENT,
OFTEN WHEN WORKERS ARE CONSULTED
IS AFTER THE FACT. UM AND YOU
KEEP IN MIND. DEPARTMENT HEADS
AND DIRECTORS ARE GENERALLY VERY
FAR REMOVED FROM PERFORMANCE OF
SPECIFIC FUNCTIONS. UM AND AS A
RESULT, OFTEN WHEN NEW
TECHNOLOGIES ARE ROLLED OUT,
THERE CAN BE A LOT OF F. THERE
CAN BE MORALE PROBLEMS AND
THERE'S NÓ WAY FOR ADJUSTMENT
AND RECALIBRATION IF THE
TECHNOLOGY ISN'T ACTUALLY
PERFORMING THE FUNCTION IT'S
INTENDED TO SERVE. UM AND SO
THIS CAN ALL BE AVOIDED BY
MAKING A I AND THE
IMPLEMENTATION OF A I A
MANDATORY SUBJECT OF BARGAINING.
UM YOU KNOW TWO, FOR EXAMPLE,
TWO PANELISTS MENTIONED THE
POTENTIAL FOR UPSKILLING CIVIL
SERVANTS. UM BUT THAT RAISES A
LOT OF QUESTIONS, RIGHT? UH,
UPSCALING REQUIRES MONEY FOR
TRAINING. IT REQUIRES FUNDING
FOR SALARY INCREASES AND ARE
THERE ENOUGH UPSKILLED POSITIONS
TO SATISFY THE CURRENT WORKFORCE
, SO ALL OF THAT CAN BE
ADDRESSED IN BARGAINING. UM AND
YOU KNOW, LASTLY, THE TRAINING
OF A I MODELS DOES PRODUCE A LOT
OF CARBON EMISSIONS. SO IN AN
IDEAL WORLD, UM THE A I MODEL
WOULD BE ABLE TO REDUCE AT LEAST
THE SAME AMOUNT OF CARBON, BUT
WE DON'T KNOW IF THAT'S POSSIBLE
AND SO AGAIN, ANOTHER WAY TO
MAKE SURE THAT THE TRADING OF AN
A I MODEL IS WORTH THE PUBLIC'S
INVESTMENT IS TO INCLUDE THE
RANK AND FILE MEMBERS IN THE
BARGAINING AND PROCUREMENT
PROCESS BECAUSE THEY ARE BEST
SUITED TO DETERMINE WHETHER OR
NOT THE PUBLIC IS ACTUALLY GOING
TO THANK YOU. THANK YOU. EX
SPEAKER. THANK YOU, UH, CHAIRS
AND SENATORS. MY NAME IS
CHRISTOPHER NIELSEN. I'M THE
DIRECTOR OF EDUCATION FOR THE
CALIFORNIA NURSES ASSOCIATION,
WHERE I ALSO WORK ON A I AND
TECHNOLOGY POLICY. THANK YOU FOR
CONVENING THIS REALLY IMPORTANT
HEARING. NURSES EMBRACE AND
REGULARLY MASTER SKILL ENHANCING
TECHNOLOGIES THAT IMPROVE
PATIENT CARE QUALITY. BUT MANY
OF THE A I SYSTEMS THAT ARE
CURRENTLY ROLLING OUT IN
HOSPITALS ACROSS THE STATE
THREATENED TO DESKILL NURSES
WORK, DEGRADE QUALITY OF CARE
AND PERPETUATE HEALTH INEQUITIES
IN OUR COMMUNITIES. NURSES URGE
YOU TO DEVELOP POLICY FRAMEWORKS
BASED ON THE PRECAUTIONARY
PRINCIPLE, WE KNOW THAT, YOU
KNOW, ADVERSE EVENT REPORTING
AUDITING THESE POST MARKET
RESPONSES ARE NECESSARY, BUT
INSUFFICIENT. YOU KNOW, WE NEED,
UM, A ROBUST REGULATIONS THAT
REQUIRE DEVELOPERS AND DEERS TO
DEMONSTRATE THE A. I SYSTEMS ARE
SAFE, EQUITABLE AND EFFECTIVE.
THEY ACTUALLY IMPROVE QUALITY OF
CARE BEFORE THEY ARE DEPLOYED IN
HEALTH CARE SETTING. WE'VE HEARD
ABOUT, YOU KNOW THE INACCURACIES
AND BIASES THAT PLAGUE MANY
CURRENT A I SYSTEMS. CNA
RECENTLY POLLED NURSES AND FOUND
THAT 24% ARE REGULARLY PROMPTED
BY DECISION SUPPORT ALGORITHMS
TO TAKE ACTIONS THAT ARE
ACTUALLY NOT IN THE BEST
INTERESTS OF THEIR PATIENTS. AND
YET ONLY 17% OF NURSES REPORT
HAVING THE ABILITY TO OVERRIDE
THOSE SYSTEMS, SO NURSES URGE
YOU TO PURSUE POLICIES THAT
SAFEGUARD NURSES, CLINICAL
JUDGMENT AND THE CLINICAL
JUDGMENT AND PROFESSIONAL
AUTONOMY OF OTHER HEALTH CARE
WORKERS. AND POLICIES THAT
STRENGTHEN OUR VOICE IN
DECISIONS ABOUT HOW OR EVEN
WHETHER A, I OUGHT TO BE
IMPLEMENTED IN PARTICULAR USE
CASES. èC N A IS HAPPY TO
DISCUSS THESE, UH, YOU KNOW
CONCERNS FURTHER WITH YOU, AND
WE LOOK FORWARD TO WORKING WITH
YOU TO DEVELOP A I POLICIES THAT
PUT PATIENTS AND WORKERS FIRST.
THANK YOU. THANK YOU VERY MUCH.
MISTER SPEAKER. MORNING. MORNING
MR CHAIRS MEMBERS. THANK YOU SO
MUCH FOR THIS VERY, VERY
IMPORTANT HEARING. UM AND IT IS
SO TIMELY AND SO IMPORTANT
BECAUSE THE BENEFITS OR THE
HARMS OF GEN. A I AND OTHER
TECHNOLOGIES ON THE PUBLIC ON
WORKERS ARE REALLY GOING TO
DEPEND ON THE GUARD RAILS THAT
YOU ALL PUT IN PLACE AND THERE
CAN BE BENEFITS WITH STRONG
GUARDRAILS. UM AND I DON'T THINK
YOU NECESSARILY NEED A WHOLE
BUNCH OF IN THE INTRICACIES OF A
I TO DO THAT. I THINK IT HAS
BEEN SAID SEVERAL TIMES THAT
HAVING POLICIES THAT ARE HUMAN
CENTERED THAT ARE WORKER
CENTERED THAT ARE CENTERED ON
THE PUBLIC THAT WORKERS SERVE
ARE A GOOD WAY TO DEVELOP THOSE
GUARD RAILS. UM, GOING BACK TO
WHAT, UM M BARRERO AND MS BARN,
BERNHARD SAID ABOUT HUMAN IN THE
LOOP OR WORKER IN THE LOOP. IT
SHOULD REALLY BE THAT HUMANS ARE
AT THE CENTER OF ALL OF THE
POLICIES. MOVE FORWARD, AND MS
BARRERO SAID. VERY ELOQUENTLY.
WORKERS NEED TO BE AT THE
BEGINNING OF WHAT TECHNOLOGY IS
DEVELOPED. WHAT TECHNOLOGY THE
STATE PROCURES AND DEPLOYS IN
ORDER TO PROVIDE THE WORK.
IMPROVE THE WORK THAT THEY DO.
UM AND IT WAS MENTIONED THAT
THERE'S A NEED FOR TRAINING FOR
UPSKILLING OF WORKERS. REALLY I
THINK ALSO, THERE IS A NEED FOR
WORKERS WHO HAVE THE EXPERTISE
TO TRAIN THE A. I UM, AND TO
TRAIN IT TO BE AN EFFECTIVE TOOL
BECAUSE THAT IS THE MOST
IMPORTANT THAT WORKERS STAY AT
THE CENTER AND THAT A I AND
TECHNOLOGY IS A TOOL THAT THEY
CAN USE TO MAKE WORK MORE
EFFICIENT, SAFER, UM, MORE
SKILLED AND OVERALL, BETTER,
BOTH FOR WORKERS AND FOR THE
PUBLIC. UM AND THEN ALSO,
WORKERS REALLY NEED TO HAVE
OVERSIGHT, ESPECIALLY IN THE
PUBLIC SECTOR OF CONSEQUENTIAL
DECISIONS. IN THE PUBLIC SECTOR
WORKERS ARE THE ONES MAKING THE
DECISIONS ABOUT MEDICAL
ELIGIBILITY ABOUT WHETHER èC P
S¼ IS GONNA TAKE YOUR KIDS AWAY.
UM THEY ARE MET PERSON, THE 988
MENTAL HEALTH AND SUICIDE
HOTLINE. THEY ARE DECIDING ABOUT
PEOPLE'S UNEMPLOYMENT BENEFITS.
PEOPLE IN THE MOST VULNERABLE
SITUATIONS ARE COMING TO THE
STATE FOR SERVICES OR TO LOCAL
GOVERNMENT FOR SERVICES. WORKERS
ARE VERY SKILLED IN THIS. THEY
NEED TO BE THE ONES WHO HAVE
OVERSIGHT. IF IS USED IN ANY OF
THOSE DECISIONS OR THE STATE MAY
DECIDE. YOU KNOW WHAT A I SHOULD
NOT HAVE A ROLE IN THIS. WE NEED
TO HAVE WORKERS WHO MAKE THESE
DECISIONS. THIS DATA IS TOO
SENSITIVE. WE DON'T WANT THIS TO
BE PUT IN AND USED TO TRAIN A I
SO THAT'S JUST SOME OF OUR OUR
RECOMMENDATIONS. THANK YOU SO
MUCH. THIS WAS A FANTASTIC
HEARING. APPRECIATE YOUR
COMMENTS. NEXT SPEAKER, PLEASE.
GOOD MORNING, CHAIRMAN DODD,
CHAIRMAN PADILLA AND MEMBER OF
THE COMMITTEE. THANK YOU FOR THE
OPPORTUNITY. UH, MEMBERS OF THE
CO, UH, COMMITTEES. THANK YOU
FOR THE OPPORTUNITY TO PROVIDE
PUBLIC COMMENT. UH, MY NAME IS
SAMANTHA GORDON. I'M THE CHIEF
PROGRAM OFFICER AT TECH EQUITY
COLLABORATIVE. OUR MISSION IS TO
RAISE PUBLIC CONSCIOUSNESS ABOUT
ECONOMIC EQUITY ISSUES THAT
RESULT FROM THE TECH INDUSTRY
PRODUCTS AND PRACTICES AND
ADVOCATE FOR CHANGE THAT ENSURES
THAT TEXT EVOLUTION CAN BENEFIT
EVERYONE. THANK YOU FOR YOUR
LEADERSHIP IN BRINGING US
TOGETHER. THIS WAS A HEARING. UM
THE PUBLIC SECTOR IS, YOU KNOW,
AS WE'VE DISCUSSED, UNIQUELY
POSITIONED TO SET A STANDARD
THAT HELPS SHAPE THE TECH
INDUSTRY'S PRACTICES AND
PRODUCTS TO ENSURE THAT THEY ARE
BENEFICIAL TO SOCIETY. WHAT
DISTINGUISHES THE PUBLIC SECTOR
FROM THE PRIVATE ACTORS WHO
DEVELOP AND SELL THIS TECHNOLOGY
IS THAT THE PUBLIC SECTOR IS
RESPONSIBLE FOR PROVIDING
SERVICES TO EVERYONE AND
NAVIGATING A COMPLEX SET OF
REQUIREMENTS AND USE CASES TO
ENSURE THAT THE PUBLIC HAS EQUAL
ACCESS, FAIR TREATMENT AND THE
ABILITY TO PURSUE RECOURSE FOR
HARM. BECAUSE OF THE UNIQUE ROLE
THE PUBLIC SECTOR PLAYS IN
SERVING ALL PEOPLE. IT IS ALSO
UNIQUELY REQUIRED TO SET A HIGH
STANDARD FOR THE ADOPTION OF
TECHNOLOGY ALREADY WE'VE SEEN
THE WAY IN WHICH HARM CAN BE
EXACERBATED THROUGH THE ADOPTION
OF EMERGING TECHNOLOGIES IN THE
PUBLIC SECTOR WITHOUT ADEQUATE
GUARD RAILS OR A DEEP
UNDERSTANDING OF THE CONTEXT IN
WHICH TECHNOLOGY IS ENTERING. IN
FLORIDA, AN INVESTIGATION FOUND
THAT THE SCORES GIVEN TO JUDGES
DURING SENTENCING HEARINGS VIA
RISK ASSESSMENT PRODUCT WERE
SIGNIFICANTLY INACCURATE. A
PROPUBLICA STUDY FOUND THAT ONLY
20% OF THE PEOPLE PREDICTED TO
COMMIT VIOLENT CRIMES ACTUALLY
WENT ON TO DO SO. ADDITIONALLY
THEY FOUND THAT THE ALGORITHM
FALSELY FLAGGED BLACK DEFENDANTS
AS FUTURE CRIMINALS WRONGLY
LABELING THEM THIS WAY AT NEARLY
TWO TIMES THE RATE OF WHITE
DEFENDANTS IN MICHIGAN, NEARLY
40,000 PEOPLE WERE WRONGLY
ACCUSED OF UNEMPLOYMENT
INSURANCE FRAUD, AN ALGORITHM
BETWEEN 2013 AND 2015 1 WOMAN
SHARED THAT AFTER LOSING HER JOB
, THE STATE'S U I SYSTEM CLAWED
BACK HER BENEFITS. SHE LOST HER
APARTMENT. SHE HAD TO MOVE IN
WITH RELATIVES OUT OF THE STATE
AND WAS UNABLE TO CARE FOR HER
AILING MOTHER LAST MONTH,
JANUARY 2024 NEARLY A DECADE
LATER, 3000 OF THOSE 40,000
PEOPLE WERE AWARDED SOME
RESTITUTION OF APPROXIMATELY
DOLLARS EACH. AS YOU CAN IMAGINE
, THESE INCIDENTS ARE NOT
ISOLATED. THERE ARE MORE DO THE
CENTRALITY OF THE PUBLIC SECTOR
AND SOME OF THE MOST DIFFICULT
AND VULNERABLE MOMENTS OF OUR
LIVES. WE URGE THE COMMITTEE TO
SET ROBUST STANDARDS THAT PUT
PEOPLE FIRST IN THE DESIGN AND
DEPLOYMENT OF TECHNOLOGY IN THE
PUBLIC SECTOR. PEOPLE HAVE
ITERATED A LOT OF PRINCIPLES
TODAY. MANY OF THEM ARE GREAT.
UM, WE WOULD ECHO THEM, ENSURING
A CLEAR ROLE FOR WORKERS AND THE
PUBLIC AND THE DESIGN,
DEVELOPMENT AND DEPLOYMENT OF
TECHNOLOGY DEVELOPMENT OF ROBUST
STANDARDS TO REDUCE HARM PRIOR
TO THE CONTRACTING OR ADOPTION
OF TECHNOLOGY, INVESTING IN THE
PUBLIC SECTOR WORKFORCE AND
ENSURING WE HAVE TECHNICAL
CAPACITY AND ABILITY TO PROVIDE
A HUMAN IN THE LOOP FOR THESE
IMPORTANT PROCESSES AND
DECISIONS REQUIRING TRANSPARENCY
AND ACCOUNTABILITY FOR THESE
SYSTEMS AND DEVELOPING CLEAR
PROCESSES FOR OVER SITE
MONITORING AND EVALUATION OF
THESE TOOLS AND OUTLINING CLEAR
BENCHMARKS TO END CONTRACTS WHEN
THE TECHNOLOGY PS, HARMFUL OR
INEFFECTIVE. THE PUBLIC SECTOR
HAS BEEN DELIBERATELY STARVED OF
RESOURCES FOR NEARLY 50 YEARS
AND THE APPETITE FOR TECHNOLOGY
THAT CAN REDUCE BACKLOG. THE
RESULTING BACKLOGS, SPEED UP
COMPLEX PROCESSES AND IMPROVE
THE PUBLIC'S ABILITY TO
EFFICIENTLY UTILIZE GOVERNMENT
SERVICES IS RIGHTLY A REALLY
URGENT PRIORITY. BUT TO NOT FALL
PREY TO SPEED OVER QUALITY OR
THE SCIENCE FICTION THAT
PROFESSOR HO TALKED ABOUT. WE
ENCOURAGE THE COMMITTEE AND ALL
OF OUR SENATORS TO CONSIDER
THESE EXAMPLES AND PRINCIPLES TO
STRENGTHEN OUR RESILIENCE
AGAINST PRODUCTS AND PRACTICES
THAT COULD FURTHER EXACERBATE
PROBLEMS IN OUR COMMUNITIES. WE
THINK THAT THIS STRENGTHENS OUR
RESOLVE AGAINST HARMFUL USES OF
TECHNOLOGY AND ALLOWS US TO OPEN
UP TO NEW OPPORTUNITIES,
TECHNOLOGIES AND APPROACHES THAT
ARE RELIABLE, SAFE AND
BENEFICIAL FOR ALL CALIFORNIANS.
THANK YOU. THANK YOU. GOOD
AFTERNOON. GOOD AFTERNOON. GOOD
MORNING, SENATOR. YOU STILL
FEELS LIKE WE'RE ALMOST MIDNIGHT
. UM I'M HERE. CHRISTINA BOSS
HAMILTON, REPRESENTING ECONOMIC
SECURITY. CALIFORNIA ECONOMIC
SECURITY. CALIFORNIA IS A PROUD
COSPONSOR OF S¼ B 1047, WHICH IS
SENATOR WEINER'S BILL. UM WE, UH
SPECIFICALLY ARE VERY
ENTHUSIASTIC ABOUT ONE OF THE
ELEMENTS OF THE BILL, WHICH IS
THE CREATION OF CAL COMPUTES,
WHICH IS BASICALLY A PUBLIC
OPTION THAT WOULD CREATE LIKE A
PUBLIC CLOUD THAT WOULD PROVIDE
AN ALTERNATIVE TO THE CURRENT
MODEL, WHICH IS BASICALLY
CONTROLLED BY VERY FEW PLAYERS
WHO HAVE ESSENTIALLY CONTROL OF
THE MARKETPLACE, SO CREATING AN
ALTERNATIVE TO THE PRIVATE NOT
NOT THE PRIVATE CROWD CLOUD
NECESSARILY, BUT CREATING PUBLIC
ALTERNATIVES THAT WILL ALLOW FOR
INVESTMENT BY RESEARCHERS AND
ACCESS TO INFORMATION THAT
OTHERWISE FOLKS WILL BE UNABLE
TO ACCESS IS VERY IMPORTANT AND
THANK YOU FOR THIS HEARING. IT'S
LIKE THE LAST SPEAKER, ALL
RIGHT? THANK YOU. CHAIR PADILLA,
MITCH AND STAFF MITCH STEER WITH
èC F Tð, WHERE A UNION OF OVER
120,000 EDUCATORS AND CLASSIFIED
PROFESSIONALS ACROSS CALIFORNIA
AND WE WOULD LARGELY ECHO THE
COMMENTS OF MS BERNHARDT AND
OTHER LABOR ADVOCATES THAT HAVE
ALREADY SPOKEN, UH ESPECIALLY
THEIR COMMENTS AROUND THE NEED
TO INCLUDE WORKERS IN THE
DECISIONS OVER WHAT SORT OF A I
TO INTRODUCE HOW TO INTRODUCE
IT. HOW TRAINING SHOULD WORK.
IT'S A REALLY KEY POINT THAT WE
THINK NEEDS TO DRIVE THE
DISCUSSION SPECIFIC TO THE
EDUCATION WORLD. A. I IS ALREADY
EVERYWHERE. IT'S ALL OVER THE
EDUCATION SYSTEM. WORKERS ARE
BARELY BEEN INVOLVED IN THESE
DECISIONS AND IT'S CREATED A LOT
OF PROBLEMS. PROBABLY THE MOST
COMMON USE OF A I IN EDUCATION
HAS BEEN IN A VARIETY OF
SOFTWARE THAT'S USED TO ASSESS
STUDENTS AND THEN SPIT OUT
RECOMMENDATIONS ON WHAT THE
TEACHER SHOULD DO. BUT IN MANY
CASES, THIS SOFTWARE IS VERY
PROPRIETARY. THE TEACHER ISN'T
REALLY INVOLVED AT ALL THE
STUDENT GOES IN USES AN IPAD
SOME OTHER SORT OF TECHNOLOGY.
ANSWERS QUESTIONS THAT SOMETIMES
THE TEACHERS DON'T EVEN HAVE
ACCESS TO. AND THEN IT SPITS OUT
RECOMMENDATIONS. HERE IS WHAT
THIS GROUP OF STUDENTS NEED. AND
IF THAT'S WHAT THEY NEED, AND
THE TEACHER DOESN'T REALLY HAVE,
OFTEN A WHOLE LOT OF DISCRETION
OVER WHETHER OR NOT THAT MAKES
SENSE OR IF THEY AGREED WITH THE
CONCLUSIONS OF THE A. I AND THE
A. I IS JUST GETTING MUCH MORE
ADVANCED QUICKLY, FAR BEYOND OUR
CAPACITY TO DEAL WITH IT, AND SO
WE STRONGLY WOULD URGE THAT AS
WE LOOK AT THIS TECHNOLOGY, AND
AS WE LOOK AT HOW IT GETS
INTRODUCED IN THE EDUCATION
SYSTEM THAT FOCUS ON THIS
CONCEPT THAT WAS RAISED BY A LOT
OF THE SPEAKERS OF MAKING THIS
HUMAN CENTERED THAT WHERE THIS
TECHNOLOGY WORKS IS WHERE IT
TAKES A TASK OFF OF THE DESK OF
A VERY BUSY TEACHER THAT COULD
VERY EASILY BE DONE BY
TECHNOLOGY. WHETHER IT'S DATA
ENTRY PAINS ME TO SAY THAT AS
SOMEONE WHO USED TO WORK IN DATA
ENTRY PROBABLY EMPLOYEES FOR A
FEW PEOPLE, FAR FEWER PEOPLE
THAN IT DID BACK IN MY DAY WHERE
WE WANT TEACHERS TO BE SPENDING
MORE TIME WITH CHILDREN AND
EDUCATING THEM AT LESS TIME
DOING ROAD BUSY WORK THAT
TECHNOLOGY CAN DO A GREAT JOB
WITH BUT WHERE THIS TECHNOLOGY
SEEKS TO REPLACE ALL THE BEST
THINGS THAT TEACHERS AND
CLASSIFIED WORKERS DO AND ALL
THE ADVANTAGES OF HAVING AN
ACTUAL HUMAN BEING THERE
INTERACTING WITH THE STUDENT AND
CAN LEARN. MAYBE THE PROBLEM
ISN'T LONG DIVISION. THE PROBLEM
IS THINGS AT HOME AND WORKING
WITH THE STUDENT THROUGH THAT
THINGS THAT A I WON'T BE ABLE TO
DO. WHAT THIS TECHNOLOGY NEEDS
TO DO IS KIND OF MARRY THOSE TWO
CONCEPTS OF THE ADVANTAGES OF
TECHNOLOGY THE ADVANTAGES OF
HUMAN BEINGS TO CREATE THE BEST
EDUCATION SYSTEM THAT WE CAN SO
WE'RE DEFINITELY NOT ANTI
TECHNOLOGY. WE ARE JUST ANTI
INTRODUCING TECHNOLOGY THAT RUNS
THE RISK OF ELIMINATING THE
GREATEST BENEFITS OF HAVING GOOD
EDUCATION WORKERS THERE IN THE
SYSTEM SO THAT WE CAN HAVE THE
BEST SYSTEM OVERALL. THANKS.
THANK YOU. WELL, THAT CONCLUDES
PUBLIC COMMENT BEFORE WE ADJOURN
. SENATOR PADILLA ANY CLOSING
COMMENTS? THANK YOU FOR YOUR
LEADERSHIP, MR CHAIRMAN. OK I
WOULD LIKE TO FINISH BY SAYING A
VERY BIG THANK YOU TO, UH, ALL
THE, UH, SPEAKERS THAT WE HAD
TODAY. YOUR TESTIMONY WAS
AMAZING. I THINK IF I'M ANY
REFLECTION OF, UH, THE REST OF
THE SENATORS WHO ARE ON THIS
PANEL AND STAFF, I'VE LEARNED A
LOT TODAY. AND UH UH, CANDIDLY,
UH, PROBABLY HAVE AS MANY MORE
QUESTIONS AS UH, AS AS YOU
CONTINUE TO THINK THROUGH ALL
THESE DIFFERENT THINGS, BUT, UH,
THIS IS A GREAT START, AND IT'S
BEEN A GREAT CONVERSATION AND
FOR THE PUBLIC THAT JOINED US.
THANK YOU FOR YOUR
PARTICIPATION. THIS JOINT
INFORMATION HEARING IS NOW
ADJOURNED. M BERNER. WE BROUGHT
IT IN IT.