GAO-24-105645, Accessible Version, Artificial Intelligence: Actions Needed to Improve DOD’s Workforce Management

ARTIFICIAL

INTELLIGENCE

ActionsNeededto

ImproveDOD’s

Workforce

Management

Accessible Version

ReporttotheCommitteeonArmed

Services,HouseofRepresentatives

December 2023

GAO-24-105645

United States Government Accountability Office

GAOHighlights

Highlights of GAO-24-105645, a report to the

Committee on Armed Services, House of

Representatives

December 2023

ARTIFICIAL INTELLIGENCE

Actions Needed to Improve DOD’s Workforce

Management

What GAO Found

The Department of Defense (DOD) typically establishes standard definitions

of its workforces to make decisions about which personnel are to be included

in that workforce, and identifies its workforces by coding them in its data

systems. DOD has taken steps to begin to identify its artificial intelligence (AI)

workforce, but has not assigned responsibility and does not have a timeline

for completing additional steps to fully define and identify this workforce.

DOD developed AI work roles—the specialized sets of tasks and functions

requiring specific knowledge, skills, and abilities. DOD also identified some

military and civilian occupations, such as computer scientists, that conduct AI

work. However, DOD has not assigned responsibility to the organizations

necessary to complete the additional steps required to define and identify its

AI workforce, such as coding the work roles in various workforce data

systems, developing a qualification program, and updating workforce

guidance. DOD also does not have a timeline for completing these additional

steps. Assigning responsibility and establishing a timeline for completion of

the additional steps would enable DOD to more effectively assess the state

of its AI workforce and be better prepared to forecast future workforce

requirements (see figure).

Questions DOD Cannot Answer Until It Fully Defines and Identifies Its AI Workforce

DOD’s plans and strategies address some AI workforce issues, but are not

fully consistent with each other. Federal regulation and guidance state that

an agency’s Human Capital Operating Plan should support the execution of

its Strategic Plan. However, DOD’s Human Capital Operating Plan does not

consistently address the human capital implementation actions for AI

workforce issues described in DOD’s Strategic Plan. DOD also uses

inconsistent terms when addressing AI workforce issues, which could hinder

a shared understanding within DOD. The military services are also

developing component-level human capital plans that encompass AI and will

cascade from the higher-level plans. Updating DOD’s Human Capital

Operating Plan to be consistent with other strategic documents would better

guide DOD components’ planning efforts and support actions necessary for

achieving the department’s strategic goals and objectives related to its AI

workforce.

View GAO-24-105645. For more information,

contact Alissa H. Czyz at (202) 512-3058 or

[email protected].

Why GAO Did This Study

DOD has invested billions of dollars to

integrate AI into its warfighting

operations. This includes analyzing

intelligence, surveillance, and

reconnaissance data, and operating

lethal autonomous weapon systems.

DOD identified cultivating a workforce

with AI expertise as a strategic focus

area in 2018. However, in 2021 the

National Security Commission on

Artificial Intelligence concluded that

DOD’s AI talent deficit is one of the

greatest impediments to the U.S. being

AI-ready by the Commission’s target

date of 2025.

House Report 117-118, accompanying

a bill for the National Defense

Authorization Act for Fiscal Year 2022,

includes a provision for GAO to review

DOD’s AI workforce. This report

evaluates the extent to which DOD has

(1) defined and identified its AI

workforce and (2) established plans

and strategies to address AI workforce

issues, among other objectives. GAO

assessed DOD strategies and plans,

reviewed laws and guidance that

outline requirements for managing an

AI workforce, and interviewed officials.

What GAO Recommends

GAO is making three

recommendations to DOD to assign

responsibility and establish a timeline

for completing the additional steps to

define and identify its AI workforce;

and update its Human Capital

Operating Plan to be consistent with

key department strategic documents

related to AI workforce issues. DOD

partially concurred with the

recommendations, offering revisions.

GAO made clarifications, as described

in the report.

Page i GAO-24-105645 Artificial Intelligence

Contents

GAO Highlights ii

Why GAO Did This Study ii

What GAO Recommends ii

What GAO Found ii

Letter 1

Background 3

DOD Uses a Variety of Tools to Recruit AI Personnel 14

DOD Has Taken Some Steps to Define and Identify Its AI

Workforce, but Has Not Completed All Necessary Actions 16

DOD Plans and Strategies Address Some AI Workforce Issues

but Are Not Fully Consistent 25

Conclusions 30

Recommendations for Executive Action 31

Agency Comments and Our Evaluation 31

Appendix I: Objectives, Scope, and Methodology 35

Appendix II: Department of Defense Cyber Workforce Framework 39

Appendix III: Comments from the Department of Defense 42

Text of Appendix III: Comments from the Department of Defense 45

Appendix IV: GAO Contact and Staff Acknowledgments 48

GAO Contact 48

Staff Acknowledgments 48

Tables

Table 1: Additional Steps to Fully Define and Identify the Artificial

Intelligence (AI) Workforce, as Identified by the CDAO 22

Table 2: Examples of Tasks and Knowledge, Skills, and Ability

Statements for the Artificial Intelligence (AI)/Machine

Learning Specialist Work Role 40

Figures

Questions DOD Cannot Answer Until It Fully Defines and

Identifies Its AI Workforce ii

Figure 1: Example of the Community of Stakeholders Engaged in

AI Development 4

Page ii GAO-24-105645 Artificial Intelligence

Figure 2: Uses of Artificial Intelligence (AI) at the Department of

Defense 5

Accessible text for Figure 2: Uses of Artificial Intelligence (AI) at

the Department of Defense 5

Figure 3: Notional Example of Artificial Intelligence on the

Battlefield 7

Figure 4: Artificial Intelligence (AI) Work Roles 19

Accessible text for Figure 4: Artificial Intelligence (AI) Work Roles 19

Figure 5: DOD Strategies and Plans That Include Artificial

Intelligence Workforce-Related Issues 27

Accessible text for Figure 5: DOD Strategies and Plans That

Include Artificial Intelligence Workforce-Related Issues 27

Abbreviations

AI artificial intelligence

CDAO Chief Digital and Artificial Intelligence Office

DOD Department of Defense

NICE National Initiative for Cybersecurity Education

OPM Office of Personnel Management

STEM Science, Technology, Engineering, and

Mathematics

This is a work of the U.S. government and is not subject to copyright protection in the

United States. The published product may be reproduced and distributed in its entirety

without further permission from GAO. However, because this work may contain

copyrighted images or other material, permission from the copyright holder may be

necessary if you wish to reproduce this material separately.

Page 1 GAO-24-105645 Artificial Intelligence

441 G St. N.W.

Washington, DC 20548

Letter

December 14, 2023

The Honorable Mike Rogers

Chairman

The Honorable Adam Smith

Ranking Member

Committee on Armed Services

House of Representatives

The Department of Defense (DOD) has invested billions of dollars to

integrate artificial intelligence (AI) into its warfighting, support, and

business operations to maintain America’s strategic position and prevail

on future battlefields.

The 2018 DOD AI Strategy states that AI refers to

the ability of machines to perform tasks that normally require human

intelligence—for example, recognizing patterns, learning from

experiences, drawing conclusions, making predictions, or taking action—

whether digitally or as the smart software behind autonomous physical

systems.

In 2018, DOD identified cultivating a leading AI workforce

(hereafter referred to as an AI workforce) as a strategic focus area.

However, in 2021 the National Security Commission on Artificial

Intelligence concluded that DOD’s AI talent deficit represented one of the

greatest impediments to the U.S. being AI-ready by 2025.

The House Armed Services Committee report accompanying a bill for the

National Defense Authorization Act for Fiscal Year 2022 included a

GAO, Artificial Intelligence: DOD Should Improve Strategies, Inventory, Process, and

Collaboration Guidance, GAO-22-105834 (Washington, D.C.: Mar. 30, 2022); DOD,

Summary of the 2018 Department of Defense Artificial Intelligence Strategy: Harnessing

AI to Advance Our Security and Prosperity (2018); and GAO, Artificial Intelligence: Status

of Developing and Acquiring Capabilities for Weapon Systems, GAO-22-104765

(Washington, D.C.: Feb. 17, 2022).

Department of Defense, Summary of the 2018 Department of Defense Artificial

Intelligence Strategy.

This report uses “AI workforce” to refer to both the civilian and military workforces, unless

otherwise specified.

The John S. McCain National Defense Authorization Act for Fiscal Year 2019 established

the National Security Commission on Artificial Intelligence to examine AI’s impact on

issues such as national security and defense. To be AI-ready by 2025, DOD must

establish a common digital infrastructure, a digitally literate workforce, and modern AI-

enabled business practices. National Security Commission on Artificial Intelligence, Final

Report (2021).

Letter

Page 2 GAO-24-105645 Artificial Intelligence

provision that we assess DOD’s AI workforce.

This report (1) describes

the tools DOD uses to recruit personnel to meet its AI workforce needs,

and evaluates the extent to which DOD (2) has defined and identified its

AI workforce and (3) has established plans and strategies that address AI

workforce issues.

To address these objectives, we reviewed relevant laws, regulations,

executive orders, and DOD and military service plans and strategies that

outline goals and processes related to the AI workforce. We interviewed

and collected documentation from officials in DOD and the Office of

Personnel Management (OPM). We also interviewed officials and

collected documentation from the military services and six DOD and

Defense Intelligence Enterprise organizations to ensure we gathered

information from organizations that conduct AI work.

We selected these

organizations based on our analysis of documents and the importance of

AI to the organization.

Although our sample is non-generalizable, the

information obtained from officials representing the six selected

organizations offered useful insights and perspectives on AI workforce

issues in the DOD and DOD’s efforts to address them. See appendix I for

additional details on our objectives, scope, and methodology.

We conducted this performance audit from January 2022 to December

2023 in accordance with generally accepted government auditing

standards. Those standards require that we plan and perform the audit to

obtain sufficient, appropriate evidence to provide a reasonable basis for

our findings and conclusions based on our audit objectives. We believe

that the evidence obtained provides a reasonable basis for our findings

and conclusions based on our audit objectives.

H.R. Rep. No. 117-118, at 66 (2021).

The military services include the Army, Marine Corps, Navy, Air Force, and Space Force.

The six DOD and Defense Intelligence Enterprise organizations include the Defense

Advanced Research Projects Agency, U.S. Army Combat Capabilities Development

Command Army Research Laboratory, Department of the Air Force-Massachusetts

Institute of Technology AI Accelerator, National Security Agency, National Geospatial-

Intelligence Agency, and the intelligence element of the Air Force.

The Defense Intelligence Enterprise includes the Defense Intelligence Agency, National

Security Agency, National Geospatial-Intelligence Agency, National Reconnaissance

Office, and intelligence elements of the Army, Navy, Marine Corps, Air Force, and Space

Force.

Letter

Page 3 GAO-24-105645 Artificial Intelligence

Background

AIandItsImportancetoNationalSecurity

National Security Commission on AI

Americans have not yet grappled with just

how profoundly the AI revolution will impact

our national security. It is no secret that

America’s military rivals are integrating AI

concepts and platforms to challenge the

United States’ decades-long technology

advantage. Despite exciting experimentation

and a few small AI programs, the U.S.

government is a long way from being “AI-

ready.”

Source: National Security Commission on Artificial

Intelligence, Final Report (Mar. 1, 2021). I GAO-24-105645

DOD has a long history of working with AI and investing in its research

and development. Since the late 1950s, the Defense Advanced Research

Projects Agency has engaged in research aimed at applying AI principles

to defense challenges. In 1981, the Navy established its Center for

Applied Research in AI. More recently, in 2017 DOD established the

Algorithmic Warfare Cross-Functional Team, known as Project Maven, to

develop AI projects and facilitate their placement into permanent DOD

programs, according to DOD officials.

In July 2017 we convened a Forum on AI to consider the policy and

research implications of AI’s use in four areas with the potential to

significantly affect daily life—cybersecurity, automated vehicles, criminal

justice, and financial services. Twenty-three members of industry,

government, academia, and nonprofit organizations participated in the

forum. They identified accessing requisite human capital as one of four AI

challenges facing the nation.

In June 2021 we published an AI accountability framework that identifies

key practices to help ensure accountability and responsible use of AI by

federal agencies and other entities involved in the design, development,

deployment, and continuous monitoring of AI systems.

The AI

GAO, Artificial Intelligence: Emerging Opportunities, Challenges, and Implications,

GAO-18-142SP (Washington, D.C.: Mar. 28, 2018).

GAO, Artificial Intelligence: An Accountability Framework for Federal Agencies and Other

Entities, GAO-21-519SP (Washington, D.C.: June 30, 2021).

Letter

Page 4 GAO-24-105645 Artificial Intelligence

accountability framework describes the broader community of

stakeholders involved in AI development, including technical stakeholders

such as data scientists, data engineers, developers, cybersecurity

specialists, program managers, and others. These technical stakeholders

are responsible for ensuring that the AI system solves the problem initially

identified, uses data sets appropriate for the problem, selects the most

suitable algorithms, and evaluates and validates the system to ensure it is

functioning as intended. The broader community of stakeholders also

includes others engaged in the development of AI, such as policy and

legal experts, subject matter experts, and individuals using the AI system

or impacted by its use. See figure 1 for an example of the community of

stakeholders engaged in AI development.

Figure 1: Example of the Community of Stakeholders Engaged in AI Development

Letter

Page 5 GAO-24-105645 Artificial Intelligence

As we reported in February 2022, DOD identified a variety of potential

warfighting and non-warfighting uses for AI across the department.

DOD’s potential AI uses in warfighting operations include analyzing

intelligence, surveillance, and reconnaissance sources; fusing data to

provide a common operating picture on the battlefield; supporting

semiautonomous and autonomous vehicles; and operating lethal

autonomous weapon systems. Potential non-warfighting uses for AI (i.e.,

support and business operations) include resolving unmatched financial

transactions, predicting maintenance needs, vetting security clearances,

and analyzing warfighter health screenings. Figure 2 shows DOD’s three

categories of potential AI uses and how they relate to risk to human life if

AI fails.

Figure 2: Uses of Artificial Intelligence (AI) at the Department of Defense

Accessible text for Figure 2: Uses of Artificial Intelligence (AI) at the Department of

Defense

· Warfighting Operations: AI used on the battlefield, such as

autonomous weapons.

GAO-22-104765.

Letter

Page 6 GAO-24-105645 Artificial Intelligence

· Support Operations: AI used to support decision makers, such as

identifying anomalies in warfighter health scans or predicting

mechanical failures in weapon platforms

· Business Operations: AI used to automate some aspects of time

consuming tasks, such as accrediting software systems for

operational deployment

Source: GAO representation of Department of Defense information. l GAO-24-105645

In February 2022 we also reported that DOD is currently pursuing AI

capabilities for warfighting that largely focus on (1) recognizing targets

through intelligence and surveillance analysis, (2) providing

recommendations to operators on the battlefield (such as where to move

troops or which weapon is best positioned to respond to a threat), and (3)

increasing the autonomy of uncrewed systems.

Several potential

applications of AI can be trained for integration into multiple weapon

platforms, such as aerial drones that are used across all the military

services. See figure 3 for a depiction of how these types of AI capabilities

can work.

GAO-22-104765.

Letter

Page 7 GAO-24-105645 Artificial Intelligence

Figure 3: Notional Example of Artificial Intelligence on the Battlefield

NOTE: The notional example depicts AI capabilities that focus on (1) recognizing targets through

intelligence and surveillance analysis, (2) providing recommendations to operators on the battlefield,

and (3) increasing the autonomy of uncrewed systems.

Additionally, in March 2022 we reported that the 2018 DOD AI Strategy

and related plans include some, but not all, characteristics of a

comprehensive strategy, and DOD lacked a high-level plan or roadmap

that captures all requirements, activities, and milestones to fully develop

its AI inventory.

We concluded that DOD would be better positioned to

help managers ensure accountability and responsible use of AI in

government programs and processes by ensuring future AI strategies and

plans include key elements of a comprehensive strategy and select

internal controls. As a result, we recommended that DOD issue guidance

GAO-22-105834.

Letter

Page 8 GAO-24-105645 Artificial Intelligence

to the Chief Digital and AI Office (CDAO), military services, and relevant

DOD organizations requiring the inclusion of all characteristics of a

comprehensive strategy in future AI strategies and associated plans and

the consideration of the key governance practices outlined in the GAO AI

accountability framework. We also recommended the CDAO in

collaboration with the Office of the Under Secretary of Defense

(Comptroller) and other entities, as appropriate, develop a high-level plan

or roadmap that captures all requirements, activities, and milestones that

support the preparation of DOD’s AI portfolio inventory and budget data.

DOD concurred with each of our recommendations. As of November

2023, DOD had not fully implemented them.

In June 2023, we reported that DOD would be better positioned to acquire

AI capabilities in a manner that accounts for the unique challenges

associated with AI by issuing department-wide AI acquisitions guidance.

WorkforceDevelopmentintheFederalGovernmentand

DOD

Defining and identifying workforces. The federal government defines

and identifies its workforces through one or more approaches. Federal

law requires OPM to publish standards and define occupations for the

federal government.

Generally, OPM develops occupational series that

include positions in a similarly specialized line of work designated by a

title and number (e.g., data science, 1560).

Occupational series

developed by OPM include computer science (1550), data science

(1560), and computer engineering (0854). DOD often defines its

workforces through OPM’s occupational series and classification

standards, according to officials with the Defense Civilian Personnel

Advisory Service.

In some instances federal laws may direct one or more federal agencies

on how they should define and identify specific workforces. For example,

Congress passed the Defense Acquisition Workforce Improvement Act in

1990 to formally define DOD’s acquisition workforce and directed DOD to

GAO, Artificial Intelligence: DOD Needs Department-Wide Guidance to Inform

Acquisitions, GAO-23-105850 (Washington, D.C.: June 29, 2023).

See section 5105 of title 5 United States Code; OPM, The Classifier’s Handbook (Aug.

1991) and Introduction to the Position Classification Standards (Aug. 1991, revised Aug.

2009).

OPM, Introduction to the Position Classification Standards.

Letter

Page 9 GAO-24-105645 Artificial Intelligence

identify the positions that DOD must include in the workforce.

According

to DOD officials and our analysis of DOD guidance, DOD defines its

workforces by establishing a standard definition that describes the

workforce in sufficient detail such that decisions can be made about

which personnel are included in the workforce. DOD identifies its

workforces by coding the workforce in DOD’s various manpower data

systems (which we refer to in this report as “workforce data systems”).

In other instances, the federal government may use work roles—the

specialized sets of tasks and functions requiring specific knowledge,

skills, and abilities—to assist in defining and identifying workforces.

For

example, DOD used the work roles from its DOD Cyber Workforce

Framework to help define and identify its cyber workforce.

OPM

provided guidance to support the framework for coding and identifying

cyber positions.

Workforce and human capital planning. Federal regulation and OPM

provide guidance and direction for workforce planning to federal

agencies, including DOD. Federal regulation requires federal agencies to

include strategic planning and alignment in their human capital planning.

This requirement is intended to ensure human capital programs are

consistent with agency mission, goals, and objectives. Federal regulation

also states that federal agencies’ human capital planning should promote

Defense Acquisition Workforce Improvement Act, Pub. L. No. 101-510, § 1201-1211

(1990) (relevant sections codified as amended at 10 U.S.C. §§ 1701-1764).

According to Defense Civilian Personnel Advisory Service officials and our analysis of

DOD guidance, DOD codes civilian personnel using the OPM occupational series and

classification standards. DOD codes military personnel using codes unique to each

military service that represent the primary military occupation of a service member. Less

frequently, for the cyber workforce and other workforces, DOD codes personnel using

codes unique to each work role. DOD Instruction 1336.05, Automated Extract of Active

Duty Military Personnel Records (July 28, 2009; Incorporating Change 3, Effective August

26, 2021) and DOD Instruction 8140.02, Identification, Tracking, and Reporting of

Cyberspace Workforce Requirements (Dec. 21, 2021).

Federal Cybersecurity Workforce Assessment Act of 2015, Pub. L. No. 114-113, § 302

(2015). Codified at 5 U.S.C. § 301 note.

The DOD Cyber Workforce Framework establishes the department’s authoritative

lexicon based on the work an individual is performing and describes the work performed

by the cyber workforce as defined in DOD Directive 8140.01, Cyberspace Workforce

Management (Oct. 5, 2020).

5 C.F.R. § 250.203 (2023).

Letter

Page 10 GAO-24-105645 Artificial Intelligence

a high-performing workforce by identifying and closing skill gaps, and

implementing programs to attract and retain quality talent.

OPM issued a five-step workforce planning model to help federal

agencies implement a systematic process for identifying and addressing

the gaps between the current workforce and future workforce

requirements.

The planning model helps determine the current

workforce resources and how they will evolve over time, including the

numbers, kinds, and location of workers needed to accomplish the

agency’s strategic requirements.

Furthermore, DOD issued its own guidance for workforce and human

capital planning. According to DOD policy, its strategic human capital

planning provides a prescriptive methodology to assess the current state

of a workforce, identify skill and competency gaps and strengths, forecast

emerging and future workforce requirements, and develop strategies to

address gaps.

In addition, DOD’s workforce planning guide outlines best

practices and provides a template with a number of core requirements.

The department identified these best practices as essential, such as

including the workforce size as part of the current workforce analysis and

evaluating future workforce requirements as part of the future workforce

analysis.

DOD has established forums to develop and implement workforce-related

policies and programs. For example, DOD established the Deputy’s

Workforce Council as the department’s principal forum for programs

related to people management, personnel policy, and total force

requirements.

According to officials, offices such as the CDAO also

5 C.F.R. § 250.203 (2023).

The five steps are (1) set strategic direction, (2) conduct workforce analysis, (3) develop

workforce action plan, (4) implement and monitor workforce action plan, and (5) evaluate

and revise workforce action plan. OPM, Workforce Planning Guide (Nov. 2022).

DOD Instruction 1400.25, Volume 250, DOD Civilian Personnel Management System:

Civilian Strategic Human Capital Planning (SHCP) (June 7, 2016).

An assessment of the current and future AI workforce would address questions such as

who is and should be included in the workforce, what is the current state and future

requirements of the workforce, and which positions require personnel from the workforce.

DOD, Defense Civilian Personnel Advisory Service: Strategic Workforce Planning Guide

(May 2019).

DOD Directive 5105.79, DOD Senior Governance Framework (Nov. 8, 2021).

Letter

Page 11 GAO-24-105645 Artificial Intelligence

have leadership roles in workforce-related forums, which include the DOD

Digital Talent Management Forum.

We have identified strategic human capital management as a high-risk

area for the federal government.

In particular, we have previously found

that mission-critical skills gaps across federal agencies in fields such as

science, technology, engineering, and mathematics pose a high risk to

the nation. In addition, we previously identified leading practices for the

federal government in workforce planning and recruitment and hiring.

The leading practices include having a process to address skills and

competency gaps, as well as developing customized strategies to recruit

highly specialized and hard-to-fill positions.

2018DODAIStrategy

The 2018 DOD AI Strategy directs DOD to accelerate the adoption of AI

and emphasizes that a strong, technologically advanced department is

essential for protecting the security of the nation. In addition, the 2018

DOD AI Strategy establishes four strategic focus areas, including

cultivating a leading AI workforce.

EntitieswithRolesandResponsibilitiesfortheAI

Workforce

The following organizations and positions have responsibilities for

different aspects of the AI workforce:

GAO, High-Risk Series: Efforts Made to Achieve Progress Need to Be Maintained and

Expanded to Fully Address All Areas, GAO-23-106203 (Washington, D.C.: Apr. 20, 2023).

GAO, “Leading Practices in Human Capital Management,” accessed Apr. 5, 2023,

https://www.gao.gov/leading-practices-human-capital-management; State Department:

Additional Actions Needed to Address IT Workforce Challenges, GAO-22-105932

(Washington, D.C.: July 12, 2022); Human Capital: Transforming Federal Recruiting and

Hiring Efforts, GAO-08-762T (Washington, D.C.: May 8, 2008); and Human Capital: Key

Principles for Effective Strategic Workforce Planning, GAO-04-39 (Washington, D.C.: Dec.

11, 2003).

Letter

Page 12 GAO-24-105645 Artificial Intelligence

· OPM defines occupations for the federal government, among other

duties, in its role as the chief human resources agency and personnel

policy manager for the federal government.

· Under Secretary of Defense for Personnel and Readiness

oversees DOD’s strategic human capital planning and issues

guidance for all DOD components regarding personnel

management.

The Defense Civilian Personnel Advisory Service

develops and implements civilian human resources plans, policies,

and programs for DOD.

· CDAO serves as the Functional Community Manager for the data,

analytics, and AI workforce.

In addition, the CDAO leads and

oversees DOD’s strategy development and policy formulation for AI,

including defining the AI workforce.

The CDAO is also an Office of

the Secretary of Defense Principal Staff Assistant.

· Chief Information Officer serves as a co-chair of the Cyberspace

Workforce Management Board, which has approval authority for

adding, modifying, and removing work roles to the DOD Cyber

Workforce Framework.

OPM also governs the general classification system. 5 U.S.C. § 5105; OPM, The

Classifier’s Handbook (Aug. 1991); and OPM, “About Us,” accessed Feb. 2, 2023,

https://www.opm.gov/about-us.

DOD Instruction 1400.25, Volume 250 and DOD Directive 5124.02, Under Secretary of

Defense for Personnel and Readiness (USD(P&R)) (June 23, 2008).

Chief Digital and Artificial Intelligence Officer Memorandum, Office of the Secretary of

Defense Functional Community Manager for the Data, Analytics and Artificial Intelligence

Workforce Designation (May 2023). Functional Community Managers are responsible for

the Strategic Human Capital Planning for their assigned occupations within the civilian

workforce. DOD Instruction 1400.25, Volume 250.

Deputy Secretary of Defense Memorandum, Initial Operating Capability of the Chief

Digital and Artificial Intelligence Officer (Feb. 1, 2022). The CDAO stood up in February

2022 as the successor organization of the Joint AI Center, which integrated the Joint AI

Center, Defense Digital Services, and Chief Data Officer into one organization. Chief

Digital and Artificial Intelligence Officer, Chief Digital and Artificial Intelligence Officer

Reaches Full Operating Capability (June 1, 2022).

Principal Staff Assistants assign responsibilities and provide policy guidance to the

heads of other DOD components, among other duties. DOD Directive 5100.01, Functions

of the Department of Defense and Its Major Components (Dec. 21, 2010) (incorporating

change 1, effective Sept. 17, 2020).

The Under Secretary of Defense for Personnel and Readiness and the Principal Cyber

Advisor also serve as chairs of the Cyberspace Workforce Management Board. DOD

Directive 8140.01, Cyberspace Workforce Management (Oct. 5, 2020) and DOD Manual

8140.03, Cyberspace Workforce Qualification and Management Program (Feb. 15, 2023).

Letter

Page 13 GAO-24-105645 Artificial Intelligence

· Director of Administration and Management serves as the

department’s Performance Improvement Officer and is responsible for

the Strategic Management Plan and Annual Performance Plan.

RelatedWorkforces

In addition to DOD’s AI workforce, several other DOD workforces also

encompass AI.

Innovation workforce. DOD refers to its “innovation workforce” in a

number of documents. For example, the Deputy Secretary of Defense

memo directed the establishment of an Innovation Workforce Team in

May 2022 to develop actionable recommendations for innovation

workforce requirements and career pathways, among other tasks.

DOD

does not yet have a standard definition for its innovation workforce, but

according to officials the department considers AI a part of its innovation

workforce.

Digital workforce. DOD refers to its “digital workforce” in a number of

documents. For example, the 2019 DOD Digital Modernization Strategy

establishes cultivating talent for a ready digital workforce as a goal for the

department.

DOD does not yet have a standard definition for its digital

workforce, but according to CDAO officials the department considers AI a

part of its digital workforce. Section 909 of the National Defense

Authorization Act for Fiscal Year 2022 defines digital talent as including

positions and capabilities in, or related to, AI, data science, autonomy,

and software development.

DOD, Director of Administration & Management, “Performance,” accessed July 5, 2023,

https://dam.defense.gov/Performance-mgmt/.

DOD is working to expand the scope of the cyber workforce to include the technological

areas pertaining to AI, cloud, data and secure software development. DOD Cyber

Workforce Strategy 2023-2027 (Mar. 1, 2023).

DOD, Deputy Secretary of Defense Memorandum, Building the Innovation Workforce

(May 16, 2022).

DOD, DOD Digital Modernization Strategy (July 12, 2019).

Pub. L. No. 117–81 § 909 (2021).

Letter

Page 14 GAO-24-105645 Artificial Intelligence

DODUsesaVarietyofToolstoRecruitAI

Personnel

DOD organizations and the military services told us they use a variety of

tools to recruit AI personnel—both uniformed service members and

civilians. These tools include a mixture of human resource flexibilities and

recruitment incentives.

The following information provides examples of

how DOD uses various tools to recruit AI personnel.

· Direct hire authorities. Officials from multiple DOD organizations

and military services, including the National Geospatial-Intelligence

Agency, Army, and Air Force, said they use direct hire authorities to

recruit AI personnel. Direct hire authorities allow an agency to

expedite the typical hiring process by eliminating certain steps

traditionally required for competitive hiring. Section 3304 of title 5 of

the United States Code permits agencies to use direct hire authorities

for civilian personnel when OPM has determined that a severe

shortage of candidates or critical hiring need exists.

According to Air

Force documentation, direct hire authorities enable the Air Force to

hire the right candidate more quickly than the traditional competitive

hiring process. For example, Air Force documentation identified the

use of direct hire authorities to fill mission-critical and hard-to-fill

civilian positions, including AI personnel, as a good practice.

· Special pay authorities. Officials from multiple DOD organizations

and military services, including the Office of the Under Secretary of

Defense for Research and Engineering, Army, and Air Force, told us

they use special pay authorities to recruit AI personnel. Various

sections under title 5 of the United States Code allow agencies to use

special pay authorities that pay higher rates to employees in select

circumstances.

· Direct commissioning authority. Army officials said that the Army

Futures Command uses direct commissioning authority to recruit

For the purposes of this report, human resource flexibilities refer to tools designed to

assist managers, supervisors, and employees on personnel matters dealing with recruiting

the federal workforce.

Congress has also provided authority to the Secretary of Defense to exercise direct hire

authority to fill critical hiring needs. See, for example, section 1105 of the National

Defense Authorization Act for Fiscal Year 2017, Pub. L. No. 114-328, § 1105 (2016) and 5

U.S.C. § 9905.

5 U.S.C §§ 5305, 5333, 5377, 5753, and 5754.

Letter

Page 15 GAO-24-105645 Artificial Intelligence

military personnel performing AI-related work. Direct commissioning

authority allows the military services to commission individuals with

relevant training or experience as officers up to the rank of colonel or

captain. For example, officials said that direct commissioning authority

enables the U.S. Army Combat Capabilities Development Command

Army Research Laboratory to recruit high quality military personnel

and attract AI talent.

· Student loan repayment. Officials from multiple DOD organizations

and military services, including the National Geospatial-Intelligence

Agency, Navy, and Air Force, told us that they use student loan

repayments to recruit civilian AI personnel. Section 5379 of title 5 of

the United States Code allows agencies to make student loan

repayments on behalf of an eligible employee who signs a service

agreement for not less than 3 years with the agency in order to recruit

or retain highly qualified personnel. For example, according to Air

Force documentation the Air Force provides applicants with up to

$30,000 through the Student Loan Repayment Program for a 3-year

service commitment to hire graduates in science, engineering, cyber,

and information technology, which include AI positions.

· Scholarships-for-service. Officials from multiple DOD organizations

and military services, including the Army, Navy, and Air Force, told us

that they use scholarships-for-service to recruit civilian AI personnel.

For example, Navy officials stated that the Navy uses the Science,

Mathematics, and Research for Transformation Scholarship-for-

Service Program to attract science, technology, engineering, and

mathematics (STEM) students, which would include AI talent. This

program offers scholarships for undergraduate, master’s, and doctoral

students currently pursuing a degree in one of 24 STEM disciplines,

such as computer science and operations research. Scholars receive

full tuition, annual stipends, and internships in exchange for a service

commitment with DOD after graduation.

· Regional sites. Army officials told us they use regional sites as a tool

to recruit AI personnel. The U.S. Army Combat Capabilities

Development Command Army Research Laboratory maintains

regional sites in California, Illinois, Massachusetts, Pennsylvania, and

The Science, Mathematics, and Research for Transformation Scholarship-for-Service

Program is a one-for-one commitment; for every year of degree funding, the scholar

commits to working for a year with the DOD as a civilian employee. DOD Scholarship-for-

Service Program, “About DOD SMART Scholarship Program,” accessed Mar. 29, 2023,

https://www.smartscholarship.org/smart?id=about_smart.

Letter

Page 16 GAO-24-105645 Artificial Intelligence

Texas, which officials said is a significant incentive to attract civilian

talent from throughout the country.

· Telework. An official from the Office of the Under Secretary of

Defense for Research and Engineering told us that they use telework

to recruit AI personnel when feasible. The Telework Enhancement Act

of 2010 defines telework or teleworking as a work flexibility

arrangement under which an employee performs the duties and

responsibilities of such employee’s position, and other authorized

activities, from an approved worksite other than the location from

which the employee would otherwise work.

An official said that

Science and Technology Reinvention Laboratories use flexible

telework options to attract AI talent.

DODHasTakenSomeStepstoDefineand

IdentifyItsAIWorkforce,butHasNot

CompletedAllNecessaryActions

DOD has made efforts to define and identify its AI workforce, and plans to

complete additional steps. However, DOD has not assigned responsibility

and lacks a timeline for completing the additional steps.

DODHasMadeEffortstoDefineandIdentifyItsAI

Workforce

With the publication of the 2018 DOD AI Strategy, DOD began taking

steps to define and identify its AI workforce. The CDAO—the successor

organization to the Joint AI Center—leads DOD’s efforts with support

from a number of DOD organizations and military services. For example,

the CDAO and other staff offices within the Office of the Secretary of

Defense developed and approved five AI work roles. In addition, other

DOD organizations and the military services have made various efforts to

begin defining and identifying their AI workforces, such as the Air Force’s

establishment of AI-special experience identifiers.

Pub. L. No. 111-292, § 6501 (2010).

Organizations include Defense Intelligence Enterprise organizations, such as the

National Security Agency and the National Geospatial-Intelligence Agency.

Letter

Page 17 GAO-24-105645 Artificial Intelligence

Steps Taken by the CDAO and Other Staff Offices within the Office

of the Secretary of Defense

In its 2018 DOD AI Strategy, DOD established “cultivating a leading AI

workforce” as a strategic focus area for the department.

It also identified

several aspects of cultivating an AI workforce that needed further

development, such as streamlining approaches to develop and attract AI

talent and building a culture that embraces experimentation.

DOD also determined that it is not yet postured to compete with industry

in hiring the large numbers of experienced, top-tier AI talent needed to

build and deploy AI across the department. In response, in its 2020 DOD

AI Education Strategy DOD established certifying and tracking AI talent

as one of four key pillars for accelerating the delivery of AI capabilities.

In addition, within this strategy DOD defined six AI “archetypes,” or

categories of personnel with similar AI education and training needs. The

archetypes include categories of personnel who create AI tools, run AI

systems to support end-users, and drive culture and policy change to

enable responsible AI adoption.

In September 2021, the Joint AI Center analyzed DOD’s AI workforce

environment. Specifically, the Joint AI Center conducted and analyzed

discovery interviews with AI workforce stakeholders from 16 DOD

organizations to gather information. The interviews highlighted that DOD

lacked a consistent understanding of its AI workforce and how the

DOD, Summary of the 2018 Department of Defense Artificial Intelligence Strategy

(2018).

The other three pillars include (1) prioritize AI awareness for senior leaders, (2) create a

common foundation for DOD’s digital workforce, and (3) create a cadre of Integrated

Product Teams to deliver AI capabilities. DOD, 2020 Department of Defense Artificial

Intelligence Education Strategy (Sept. 2020). Since DOD published the education

strategy, the CDAO has been assigned as the functional community manager for the data,

analytics, and AI workforce. Chief Digital and Artificial Intelligence Officer Memorandum,

Office of the Secretary of Defense Functional Community Manager for the Data, Analytics

and Artificial Intelligence Workforce Designation.

The 2020 DOD AI Education Strategy states that DOD segmented its entire workforce

into the six AI archetypes to respond to a statutory mandate in the National Defense

Authorization Act for Fiscal Year 2020, which directed the Secretary of Defense to develop

a strategy to educate service members in relevant occupational fields on AI. See National

Defense Authorization Act for Fiscal Year 2020, Pub. L. No. 116–92, § 256 (2019).

According to the 2020 DOD AI Education Strategy, DOD used the archetypes as an

analytical framework to inform key strategy decisions, including how DOD should prioritize

and sequence new actions and investments, manage costs, and establish foundations

needed to achieve a long-term vision for the department’s digital workforce.

Letter

Page 18 GAO-24-105645 Artificial Intelligence

workforce is managed. For instance, the Joint AI Center reported that

DOD used multiple and sometimes isolated approaches to defining and

developing its AI workforce. Eleven of 16 organizations (69 percent)

participating in the interviews cited the lack of a definition for the AI

workforce as a reason for not having begun to develop AI workforce

strategies within their respective organizations.

In late 2021 and early 2022, the Joint AI Center conducted additional

analyses to assist the development of a DOD AI strategic human capital

plan.

The Joint AI Center surveyed personnel in DOD organizations and

the military services to, in part, identify which occupations may include

personnel performing AI work.

Over 24 of 93 respondents (more than 26

percent) self-identified as operations researchers. Between approximately

five and nine respondents (5 to 10 percent) each self-identified as one of

six other occupational series that included computer scientists,

meteorologists, and information technology specialists. In addition,

stakeholders from DOD organizations and the military services said that

establishing a standard definition for the AI workforce was one of their top

recommendations. CDAO officials told us that they plan to perform

additional analyses to include data and analytics in addition to AI. CDAO

officials planned to complete this work by October 2023.

As of August 2023, DOD had developed and approved five AI work roles,

which officials said will assist with coding and identifying the AI workforce

in DOD’s various workforce data systems. See figure 4 for a description

of the five AI work roles.

The Joint AI Center was integrated into the CDAO when the CDAO stood up in February

2022 as the successor organization of the Joint AI Center.

As part of the Joint AI Center’s analysis of DOD’s AI workforce environment, it had

defined AI talent as “individuals with the ability to engage in AI/machine learning or

AI/machine learning-related work tasks intended to advance, maintain, and/or execute

AI/machine learning capabilities.” However, DOD has not formally adopted this working

definition. Ninety-three of 110 civilian personnel who self-identified as performing AI work

responded to the survey. The survey results are not authoritative or generalizable,

according to DOD officials and our analysis of the information provided by DOD.

As of November 2023, the work has not been completed.

Work roles refer to specialized sets of tasks and functions requiring specific knowledge,

skills, and abilities. Federal Cybersecurity Workforce Assessment Act of 2015, Pub. L. No.

114-113, § 302 (2015). DOD added the five AI work roles to the DOD Cyber Workforce

Framework in September 2022. See app. II of this report for more information on the DOD

Cyber Workforce Framework.

Letter

Page 19 GAO-24-105645 Artificial Intelligence

Figure 4: Artificial Intelligence (AI) Work Roles

Accessible text for Figure 4: Artificial Intelligence (AI) Work Roles

1. AI/Machine Learning Specialist (623)

Designs, develops, and modifies AI applications, tools, and/or other

solutions to enable successful accomplishment of mission objectives.

2. AI Test and Evaluation Specialist (672): Performs testing, evaluation,

verification, and validation on AI solutions to ensure they are

developed to be and remain robust, resilient, responsible, secure, and

trustworthy; and communicates results and concerns to leadership.

3. AI Risk and Ethics Specialist (733): Educates those involved in the

development of AI and conducts assessments on the technical and

societal risks across the lifecycle of AI solutions from acquisition or

design to deployment and use.

4. AI Adoption Specialist (753): Facilitates AI adoption by supporting the

users of AI-enabled solutions.

5. AI Innovation Leader (902): Builds the organization’s AI vision and

plan and leads policy and doctrine formation, including how AI

solutions can or will be used.

Source: GAO analysis of Department of Defense (DOD) information. | GAO-24-105645

DOD carried out the following key activities to develop and approve the

five AI work roles:

Letter

Page 20 GAO-24-105645 Artificial Intelligence

· Developed AI work roles. DOD officials designated as subject matter

experts collaborated to develop the five work roles and associated

knowledge, skills, abilities, and tasks. For example, the AI and

machine learning specialist work role consists of designing,

developing, and modifying existing AI applications and tools.

· Administered focus groups and surveys. DOD administered focus

groups and surveys to help ensure the optimal amount of stakeholder

involvement. For example, the office of the Chief Information Officer

facilitated eight focus groups with officials from 14 DOD organizations

who were designated by DOD to be AI and data subject matter

experts to consider revising or adding AI and other related content to

the DOD Cyber Workforce Framework. DOD also administered two

post-focus group surveys.

· Reviewed AI work roles. Subject matter experts and the Cyber

Workforce Advisory Group conducted reviews of the AI work roles.

· Approved AI work roles. The Cyber Workforce Management Board

approved the five AI work roles and added them to the DOD Cyber

Workforce Framework.

Each work role includes a unique three-digit

code, which will ultimately assist DOD to code and identify its AI

workforce in DOD’s various workforce data systems. The addition of

the AI work roles to the framework was a significant step toward fully

defining and identifying the DOD AI workforce, according to CDAO

officials.

In November 2023, DOD published the 2023 DOD Data, Analytics, and

Artificial Intelligence Adoption Strategy. The strategy is intended to help

accelerate the adoption of advanced data, analytics, and artificial

intelligence technologies within DOD. Additionally, the strategy identifies

expanding digital talent management as an enabler to help sustain DOD’s

AI readiness.

The DOD Cyber Workforce Framework describes the work performed by the full

spectrum of the cyber workforce, and the framework is DOD’s authoritative reference for

identifying, tracking, and reporting DOD cyberspace positions. DOD Directive 8140.01.

The work roles define the AI work to be performed by DOD personnel through a series

of knowledge, skills, abilities, and tasks. DOD and other federal agencies can use work

roles to define workforces.

Expanding digital talent management includes increasing hiring, training, and retention

for the most critical data, analytics, and AI-related work roles. 2023 DOD Data, Analytics,

and Artificial Intelligence Adoption Strategy (Nov. 2023).

Letter

Page 21 GAO-24-105645 Artificial Intelligence

Efforts Made by Other DOD Organizations and the Military Services

Other DOD organizations and the military services have also made

various efforts to begin defining and identifying their AI workforces,

including the following examples:

· The National Geospatial-Intelligence Agency added AI-related

competencies to the personnel records within its personnel data

system, according to officials. The officials told us this enables

managers and others to identify AI personnel through informal, self-

directed word searches.

· The U.S. Army Combat Capabilities Development Command Army

Research Laboratory defined three competencies that capture most of

its AI workforce, according to officials. The officials told us that the

laboratory can partially identify personnel performing AI work by

tracking those competencies in the laboratory’s personnel database.

· The Air Force established six special experience identifiers to identify

its enlisted personnel with AI or machine learning experience or

training.

These identifiers are added to enlisted personnel records in

the personnel data system as appropriate.

· The Navy is in the process of developing and approving a robotics

rating for enlisted personnel, which will include aspects of AI, and is

capable of mapping civilian AI workers to OPM occupational groups

such as data science and engineering, according to officials.

DODPlanstoCompleteAdditionalStepstoDefineand

IdentifyItsAIWorkforce

While the CDAO leads the process to fully define and identify DOD’s AI

workforce, officials from multiple organizations told us the remaining steps

to fully define and identify the AI workforce will rely on work to be

performed by multiple DOD organizations. These organizations include

the Office of the Under Secretary of Defense for Personnel and

Organizations include Defense Intelligence Enterprise organizations, such as the

National Security Agency and the National Geospatial-Intelligence Agency.

Special Experience Identifiers provide a means to track individuals and identify positions

requiring or providing unique experience or training that would otherwise be lost in the

personnel data system. These identifiers are established when identifying training, skills,

or experience is critical to the assignment match or deployment needs and when no other

personnel data system-based identification is appropriate or available. Air Force Manual

36-2100, Military Utilization and Classification (Apr. 7, 2021).

Letter

Page 22 GAO-24-105645 Artificial Intelligence

Readiness, as well as the Office of the Chief Information Officer and the

military services. Table 1 describes the additional steps and the potential

organizations the CDAO identified as responsible for performing the

work.

Table 1: Additional Steps to Fully Define and Identify the Artificial Intelligence (AI) Workforce, as Identified by the CDAO

Steps

Work to be done

Potential responsible

organizations

Coding work roles

Load three-digit codes for the five AI work roles into

workforce data systems to facilitate identifying positions

that require AI work.

Chief Digital and AI Office (CDAO),

Under Secretary of Defense for

Personnel and Readiness, DOD

organizations, military services

Determining key identifiers for AI

work roles

Conduct DOD-wide focus groups with AI subject matter

experts to determine which of the AI tasks and knowledge,

skill, and ability statements from the AI work roles are

considered essential.

CDAO, Chief Information Officer

Updating the Cyber Workforce

Management Board charter

Update the charter to formally add the CDAO as a voting

member.

CDAO, Chief Information Officer

Updating workforce guidance

Update DOD Directive 8140.01 Cyberspace Workforce

Management and DOD Manual 8140.03 Cyberspace

Workforce Qualification and Management Program.

Among other things, the updates are expected to reflect

the expansion of the DOD Cyber Workforce Framework

and add CDAO roles and responsibilities, such as being

the office of primary responsibility for the AI and data

workforce element.

CDAO, Chief Information Officer

Developing a qualification

program

Determine the qualifications for serving in the five AI work

roles, such as education, training, and personnel

qualifications.

CDAO, Chief Information Officer,

DOD organizations, military services

Approval required from the Cyber

Workforce Advisory Group and

Cyber Workforce Management

Board

Source: GAO analysis of Department of Defense (DOD) information, to include documentation provided by, and interviews with, DOD officials. | GAO-24-105645

DOD documentation states that the data systems are already set up to accept work role codes and

perform data analytics.

In February 2023, DOD published the initial version of DOD Manual 8140.03, which provides

guidance on the Cyberspace Workforce Qualification and Management Program. For example, the

program outlines qualification standards and requirements for each work role in the DOD Cyber

Workforce Framework, which includes the AI work roles. DOD Manual 8140.03, Cyberspace

Workforce Qualification and Management Program (Feb. 15, 2023).

In addition, CDAO officials told us that before DOD will consider its AI

workforce fully defined and identified, DOD plans to align the future AI

workforce definition with the information contained in OPM’s AI

As noted later in this report, the CDAO has not yet assigned responsibility to these

organizations to take actions to complete the additional steps.

Letter

Page 23 GAO-24-105645 Artificial Intelligence

occupational series once it is published.

However, as of April 2023 OPM

had not developed an AI occupational series or updated an existing

occupational series with AI-related duties as required by statute,

according to officials.

OPM officials told us OPM conducted a study,

which found that AI work is performed in various occupational series in

the Federal government and specializations. The officials told us that

OPM determined that creating one occupational series or multiple AI

series is not conducive to individual agency needs and missions. These

officials stated that instead of creating an AI occupational series, OPM will

develop and issue classification interpretive guidance to help agencies

such as DOD identify AI work and determine how to classify positions that

perform AI work and how to qualify applicants for AI positions. According

to OPM officials, they expected to issue the interpretive guidance in fourth

quarter fiscal year 2023. However, as of first quarter fiscal year 2024,

OPM officials told us the interpretive guidance is pending stakeholder

review and final OPM approval for issuance.

CDAOHasNotAssignedResponsibilityandLacksa

TimelineforCompletingtheAdditionalSteps

CDAO has not assigned responsibility or provided a timeline to the

organizations responsible for completing the additional steps needed to

define and identify the AI workforce, although the CDAO has begun

notifying the various DOD organizations of their potential responsibilities

in completing these additional steps. For example, in 2022 and early 2023

CDAO officials said they briefed representatives from the relevant DOD

organizations to make them aware of the additional steps and their

Federal law requires OPM to publish standards and define occupations for the federal

government, including DOD. 5 U.S.C. § 5105 and OPM, The Classifier’s Handbook (Aug.

1991). GAO has ongoing work examining the extent to which federal agencies have

complied with selected federal policy and guidance on AI. This includes the requirement in

section 105 of the AI in Government Act of 2020, Pub. L. No. 116-260 (2020), for OPM to

establish an occupational series, or update and improve an existing occupational job

series, to include positions whose primary duties relate to AI. This work was expected to

be completed in fourth quarter of fiscal year 2023, according to OPM officials. However,

as of first quarter fiscal year 2024, OPM officials told us the interpretive guidance is

pending stakeholder review and final OPM approval for issuance.

Section 105 of the AI in Government Act of 2020 required the Director, OPM to establish

the occupational series not later than 18 months after enactment or June 2022.

In July 2023, OPM issued AI competencies for governmentwide use, which include

general and technical competencies that were identified through an environmental scan

for AI work. OPM Memorandum, The AI in Government Act of 2020 – Artificial Intelligence

Competencies (July 6, 2023).

Letter

Page 24 GAO-24-105645 Artificial Intelligence

potential involvement at meetings of the department’s Digital Talent

Management Forum. However, this did not include the CDAO assigning

responsibility to those organizations to take actions to complete the steps.

CDAO officials told us the CDAO continues to work with DOD

organizations to complete some of the steps related to fully defining and

identifying the department’s AI workforce. The officials also told us that as

of March 2023, the CDAO was working with the Office of the Under

Secretary of Defense for Personnel and Readiness to design a

competency model for the data scientist occupational series (1560).

However, the CDAO has not yet assigned responsibility or established a

timeline to complete the additional steps necessary to fully define and

identify the AI workforce.

The CDAO has the authority to assign responsibility and establish a

timeline to the organizations responsible for completing the steps. When

established in February 2022, the CDAO was designated as a Principal

Staff Assistant with authorities related to AI and other areas as described

in two Deputy Secretary of Defense memorandums.

As a Principal Staff

Assistant, the CDAO can assign responsibilities to the heads of other

DOD components and develop policies, among other duties.

DOD established cultivating a leading AI workforce as a strategic focus

area for the department.

Further, the DOD instruction on DOD Civilian

Personnel Management System: Civilian Strategic Human Capital

Planning provides a methodology to assess the current state of the

workforce, identify gaps, and forecast emerging and future workforce

requirements to support the DOD mission.

In March 2023, CDAO officials also stated that they were beginning the process of

developing an AI competency model. However, in November 2023, CDAO officials told us

DOD would not be developing an AI competency model, but expected to update guidance

to reflect the CDAO’s roles and responsibilities, such as being the office of primary

responsibility for the AI and Data work element. The officials also noted that areas relating

to the AI workforce are rapidly evolving.

Deputy Secretary of Defense Memorandum, Initial Operating Capability of the Chief

Digital and Artificial Intelligence Officer (Feb. 1, 2022); and Deputy Secretary of Defense

Memorandum, Role Clarity for the Chief Digital and Artificial Intelligence Officer (Feb. 1,

2022).

DOD, Summary of the 2018 Department of Defense Artificial Intelligence Strategy

(2018).

DOD Instruction 1400.25, Volume 250.

Letter

Page 25 GAO-24-105645 Artificial Intelligence

CDAO officials told us they do not want to fully define and identify the AI

workforce until OPM completes its actions related to AI work. DOD wants

to ensure the department’s definition of its AI workforce aligns with OPM’s

efforts, according to CDAO officials. However, DOD can continue working

on the additional steps before OPM completes its actions, because OPM

plans to finalize its actions well before DOD’s additional steps could be

completed. For example, in September 2023 OPM officials told us that

their actions should be complete in fourth quarter fiscal year 2023, while

DOD officials estimated that it could take until January 2025 to complete

one of their additional steps—updating workforce guidance.

Since DOD

plans to be AI-ready by 2025, the department can assign responsibility,

establish a timeline, and begin work on completing the additional steps

without delay.

Until DOD does so, it will not be able to accurately assess the current

state of its AI workforce or forecast future AI workforce requirements to

support the DOD mission, as required.

DODPlansandStrategiesAddressSomeAI

WorkforceIssuesbutAreNotFullyConsistent

DOD has published plans and strategies intended to support the

department’s strategic goals and strategic objectives, including those that

address AI workforce issues. These include the DOD Strategic

Management Plan Fiscal Years 2022–2026 (hereafter referred to as the

Agency Strategic Plan), the Annual Performance Plan Fiscal Year 2023

(hereafter referred to as the Annual Performance Plan), and the Fiscal

Year 2022–Fiscal Year 2026 DOD Civilian Human Capital Operating Plan

(hereafter referred to as the Human Capital Operating Plan).

DOD did not have estimated completion dates for the other additional steps as of July

2023. As of first quarter fiscal year 2024, OPM officials told us the interpretive guidance is

pending stakeholder review and final OPM approval for issuance.

DOD Instruction 1400.25, Volume 250.

DOD, DOD Strategic Management Plan Fiscal Years 2022–2026 (July 2022); Annual

Performance Plan Fiscal Year 2023 (July 2022); and Fiscal Year 2022–Fiscal Year 2026

Department of Defense Civilian Human Capital Operating Plan (June 2022). In March

2023, DOD updated its DOD Strategic Management Plan Fiscal Years 2022–2026 and

published the Annual Performance Plan Fiscal Year 2024. DOD, DOD Strategic

Management Plan Fiscal Years 2022–2026 (Mar. 2023).

Letter

Page 26 GAO-24-105645 Artificial Intelligence

The Human Capital Operating Plan should describe how an agency will

execute human capital elements—including strategic goals and strategic

objectives—identified within the Agency Strategic Plan and Annual

Performance Plan.

The Human Capital Operating Plan also should

serve as the strategic roadmap for numerous products and human capital

business solutions to assist DOD in managing the life cycle of its civilian

workforce.

The effectiveness of the Human Capital Operating Plan

depends largely on engaging with stakeholders such as DOD senior

management officials.

See figure 5 for a description of DOD strategies

and plans.

5 C.F.R. § 250.205 (2023) and OPM, Human Capital Operating Plan (HCOP) Guidance

Fiscal Years 2022–2026 (Dec. 2021).

The Defense Civilian Personnel Advisory Service develops DOD’s Human Capital

Operating Plan. A Defense Civilian Personnel Advisory Service official told us that DOD

reviews, approves, and updates the Human Capital Operating Plan annually.

OPM, Human Capital Operating Plan Guidance.

Letter

Page 27 GAO-24-105645 Artificial Intelligence

Figure 5: DOD Strategies and Plans That Include Artificial Intelligence Workforce-

Related Issues

Accessible text for Figure 5: DOD Strategies and Plans That Include Artificial

Intelligence Workforce-Related Issues

· National Defense Strategy: Serves as DOD’s primary strategy

document and provides a foundation for all other strategic guidance in

the department.

· Agency Strategic Plan: Presents the long-term objectives DOD hopes

to accomplish at the beginning of each new term of an administration.

· Annual Performance Plan: Defines the level of performance to be

achieved during the first two fiscal years after the plan is submitted.

Letter

Page 28 GAO-24-105645 Artificial Intelligence

· Human Capital Operating Plan: Serves as the department’s human

capital implementation document and describes how an agency will

execute the human capital elements stated within the Agency

Strategic Plan and the Annual Performance Plan.

Source: GAO analysis of Department of Defense (DOD) information. | GAO-24-105645

Federal regulation and OPM guidance state that a Human Capital

Operating Plan should support the execution of the Agency Strategic

Plan.

This includes addressing the human capital implementation

actions planned to support the strategic goals and priorities identified in

the Agency Strategic Plan and Annual Performance Plan.

However, we found that the Human Capital Operating Plan is not fully

consistent with the Agency Strategic Plan and Annual Performance Plan.

Specifically, the Human Capital Operating Plan does not consistently

address human capital implementation actions for AI-related workforce

issues, and DOD’s plans were inconsistent in how they discussed AI-

related terms.

Does not consistently address AI implementation actions. The

Human Capital Operating Plan does not consistently address human

capital implementation actions for the required strategic goals cited in the

Agency Strategic Plan that relate to AI workforce issues. For example,

Strategic Goal One—Making the Right Technology Investments and

Transforming the Future Force—includes a strategic objective to build a

strong foundation for future science and technology by investing in the

department’s workforce, laboratory, and testing infrastructure. To support

this strategic objective, the Annual Performance Plan includes a

performance measure to conduct an annual assessment of workforce,

infrastructure, and industrial base capabilities and capacity to support the

critical technology areas road maps.

However, the Human Capital

Operating Plan does not discuss annual assessments of the workforce or

reference critical technology, which includes AI.

5 C.F.R. § 250.205 (2023).

Office of Management and Budget, Circular No. A-11 Preparation, Submission, and

Execution of the Budget (Aug. 2022); 5 C.F.R. § 250.205 (2023); and OPM, Human

Capital Operating Plan Guidance.

AI is considered a critical technology according to an Office of the Under Secretary of

Defense for Research and Engineering memorandum published in February 2022. Under

Secretary of Defense for Research and Engineering, USD (R&E) Technology Vision for an

Era of Competition (Feb. 1, 2022).

Letter

Page 29 GAO-24-105645 Artificial Intelligence

In addition, Strategic Goal Four—Address Institutional Management

Priorities—includes a strategic objective to accelerate DOD’s adoption of

AI. Additionally, the Annual Performance Plan identifies AI expertise and

literacy within the department as a major management challenge to

accelerating the adoption of AI.

However, the Human Capital Operating

Plan does not reference accelerating the adoption of AI or the AI

expertise and literacy management challenge.

Inconsistent AI-related terms. DOD’s plans are also inconsistent in how

they discuss AI-related terms. Some of the military services are also

developing or planning to develop component-level human capital plans

that encompass AI and will cascade from the higher-level plans.

For example, DOD’s Agency Strategic Plan and Annual Performance

Plan both use the term “critical technology areas” when discussing the

need to conduct annual workforce assessments. Examples of DOD

critical technologies include AI, quantum science, biotechnology, and

microelectronics.

By contrast, when discussing the need to develop an

inventory of skills gaps, the Human Capital Operating Plan uses terms

such as the “innovation workforce” and “digital talent,” but does not

reference “critical technology areas.”

If DOD-wide plans do not use

consistent terminology, the proposed component level plans may also

use the same inconsistent terms when addressing AI workforce issues.

This could prevent a common department-wide understanding of AI

workforce strategic goals and objectives.

DOD, FY 2021 Annual Performance Plan FY 2019 Annual Performance Report (Jan.

2020). Major management challenge refers to programs or management functions that

have greater vulnerability to waste, fraud, abuse, and mismanagement, such as programs

and management functions we or another Inspector General have identified as high risk.

For these programs and management functions, a failure to perform well could seriously

affect the ability of an agency or the government to achieve its mission or goals. 31 U.S.C.

§ 1115(h)(5).

This includes both the most recent Human Capital Operating Plan and its immediate

predecessor (i.e., the Fiscal Year 2022-2026 DOD Civilian Human Capital Operating Plan

and the Fiscal Year 2020-2021 DOD Civilian Human Capital Operating Plan).

Under Secretary of Defense for Research and Engineering, USD (R&E) Technology

Vision for an Era of Competition (Feb. 1, 2022).

As mentioned previously, DOD does not yet have a standard definition for its innovation

or digital workforces. However, DOD officials use the undefined terms “innovation

workforce” and “digital workforce” to refer to DOD’s AI workforce, among other workforces.

Letter

Page 30 GAO-24-105645 Artificial Intelligence

A Defense Civilian Personnel Advisory Service official told us that they

believe the human capital objectives and strategies described in the

Human Capital Operating Plan align with strategic goals in the Agency

Strategic Plan. However, the Human Capital Operating Plan does not

identify or mention the strategic goals discussed above.

The next annual review of the Human Capital Operating Plan provides an

opportunity to address human capital implementation actions for AI

workforce issues and ensure the use of consistent AI terminology. Until

the Human Capital Operating Plan is fully consistent with the Agency

Strategic Plan and Annual Performance Plan, efforts to develop the

various component-level human capital plans may not fully implement

actions necessary for achieving DOD’s strategic goals and objectives.

Conclusions

Cultivating a leading AI workforce within DOD is critical to maintaining

America’s strategic position and prevailing in future battlefields. Since

2018, DOD has made organizational changes and is investing billions of

dollars to incorporate AI technology into its operations. CDAO and other

staff offices within the Office of the Secretary of Defense have taken

some steps to define and identify DOD’s AI workforce. However, DOD

has not formally assigned responsibility and does not have a timeline for

completing the additional steps required to fully define and identify its AI

workforce, such as coding the work roles in various workforce data

systems, developing a qualification program, updating workforce

guidance, and any other actions DOD may identify. Until DOD completes

these and any other steps DOD identifies as necessary to fully define and

identify the AI workforce, the department will not be able to accurately

assess the current state of its AI workforce or forecast future AI workforce

requirements to support the DOD mission, as required.

In addition, DOD has published plans and strategies that address some

AI workforce issues. However, DOD’s Human Capital Operating Plan is

not fully consistent with its Agency Strategic Plan and Annual

Performance Plan as it relates to AI workforce issues. By ensuring that

the Human Capital Operating Plan addresses implementation actions and

uses AI-related terms consistent with the Agency Strategic Plan and

DOD’s Human Capital Operating Plan does not reference the strategic goals discussed

above: Strategic Goal One Making the right technology investments and Strategic Goal

Four Address institutional management priorities.

Letter

Page 31 GAO-24-105645 Artificial Intelligence

Annual Performance Plan, DOD will be better positioned to achieve the

department’s strategic goals and objectives related to AI throughout DOD

and its components.

RecommendationsforExecutiveAction

We are making the following three recommendations to DOD:

The Secretary of Defense should ensure the CDAO assigns responsibility

to complete the additional steps necessary to fully define and identify

DOD’s AI workforce. (Recommendation 1)

The Secretary of Defense should ensure the CDAO establishes a timeline

for additional steps necessary to fully define and identify the AI workforce.

(Recommendation 2)

The Secretary of Defense should ensure that the Under Secretary of

Defense for Personnel and Readiness updates the Human Capital

Operating Plan to be consistent with the Agency Strategic Plan and

Annual Performance Plan relating to AI workforce issues in the next

annual review. This should include (1) addressing the human capital

implementation actions planned to support the strategic goals and

priorities identified in the Agency Strategic Plan and Annual Performance

Plan and (2) ensuring the use of consistent AI terminology.

(Recommendation 3)

AgencyCommentsandOurEvaluation

We provided a draft of this report to DOD and OPM for review and

comment. DOD provided written comments, which are reproduced in

appendix III. DOD partially concurred with each of our three

recommendations. DOD and OPM provided technical comments, which

we incorporated into our report as appropriate.

DOD partially concurred with our first two recommendations that the

CDAO should assign responsibility and establish a timeline for completing

additional steps to fully define and identify DOD's AI workforce. In its

written comments, DOD stated there is a conflation of workforce definition

and identification processes for work roles compared to occupational

series, and that there are inaccuracies regarding the order of operations,

Letter

Page 32 GAO-24-105645 Artificial Intelligence

and proposed revisions to the examples highlighted in our

recommendations.

We do not agree that we conflated workforce definition and identification

processes for work roles compared to occupational series. We based our

analysis on OPM and DOD guidance. DOD did not describe or elaborate

on how the processes might be conflated in its written response or in its

technical comments provided along with the response to our draft report.

However, based on DOD’s technical comments and updated information

on DOD’s planned actions obtained through follow-up discussions with

CDAO officials, we made adjustments to our report and we also removed

the specific examples of actions from our recommendations to provide

DOD flexibility to identify the remaining actions needed to fully define and

identify the AI workforce.

DOD also proposed that we revise the wording of the first two

recommendations to include the Under Secretary of Defense for

Personnel and Readiness as a coordinating office. We applaud DOD’s

emphasis on stakeholder coordination and recognize the value that

coordination with the Under Secretary of Defense for Personnel and

Readiness and others, such as the Chief Information Officer, can add to

the department’s efforts to define and identify the AI workforce. However,

we did not make this change to the recommendations because of the

CDAO’s existing ability to coordinate with the Under Secretary of Defense

for Personnel and Readiness and others in its efforts to define and

identify the AI workforce.

We continue to believe that DOD should assign responsibility and

establish a timeline for completing the additional steps necessary to fully

define and identify its AI workforce. Until these steps are completed, the

department will not be able to accurately assess the current state of its AI

workforce or forecast future AI workforce requirements to support the

DOD mission.

DOD partially concurred with our third recommendation and stated that

the department recently completed a Human Capital Operating Plan

Letter

Page 33 GAO-24-105645 Artificial Intelligence

Refresh for fiscal years 2022 through 2026.

DOD proposed that we

revise the wording of the recommendation by adding language about

stakeholder engagement. We appreciate DOD’s emphasis on engaging

with stakeholders and, as we noted in our report, the effectiveness of the

Human Capital Operating Plan depends largely on engaging with

stakeholders. However, we did not modify our recommendation because

of the existing ability of the Under Secretary of Defense for Personnel and

Readiness to engage with stakeholders.

We continue to believe that DOD will be better positioned to achieve its

strategic goals and objectives related to AI if it ensures that its Human

Capital Operating Plan addresses implementation actions and uses AI-

related terms consistent with the Agency Strategic Plan and Annual

Performance Plan.

We are sending copies of this report to the appropriate congressional

committees, the Secretary of Defense, the Deputy Secretary of Defense,

the Under Secretary of Defense for Personnel and Readiness, the Under

Secretary of Defense for Research and Engineering, the Chief

Information Officer, the CDAO, Secretaries of the Air Force, Army, and

Navy, and the Director of OPM. In addition, the report is available at no

charge on the GAO website at https://www.gao.gov.

If you or members of your staff have any questions regarding this report,

please contact me at (202) 512-3058 or [email protected]. Contact points

for our Offices of Congressional Relations and Public Affairs may be

found on the last page of this report. Key contributors to this report are

listed in appendix IV.

On November 30, 2023, DOD provided us a copy of the refresh, which is the action plan

and metric updates to the Fiscal Year 2022–Fiscal Year 2026 DOD Civilian Human

Capital Operating Plan. The refresh includes an initiative to initiate pilots for data,

analytics, and artificial intelligence talent acquisition and a milestone to develop,

implement, and assess a recruitment marketing campaign to promote civilian

cyber/artificial intelligence career opportunities with DOD, among other things. DOD

Human Capital Operating Plan Fiscal Year 2024-2025.

Letter

Page 34 GAO-24-105645 Artificial Intelligence

Alissa H. Czyz

Director, Defense Capabilities and Management

Appendix I: Objectives, Scope, and

Methodology

Page 35 GAO-24-105645 Artificial Intelligence

AppendixI:Objectives,Scope,

andMethodology

In a report accompanying a bill for the National Defense Authorization Act

for Fiscal Year 2022, the House Armed Services Committee included a

provision that we assess the Department of Defense’s (DOD) artificial

intelligence (AI) workforce.

This report (1) describes the tools DOD uses

to recruit personnel to meet its AI workforce needs, and evaluates the

extent to which DOD (2) has defined and identified its AI workforce and

(3) has established plans and strategies that address AI workforce issues.

To address these objectives, we reviewed relevant laws, regulations, and

DOD and military service plans and strategies that outline goals and

processes related to the AI workforce. We also interviewed and collected

documentation from DOD and Office of Personnel Management (OPM)

officials.

In addition, we interviewed officials and collected documentation

from the military services, three DOD organizations, and three Defense

Intelligence Enterprise organizations to ensure we gathered information

from organizations that conduct AI work.

We included three of the nine

organizations from the Defense Intelligence Enterprise because the

Intelligence Community identifies itself as a leader in developing and

using technology crucial to the U.S. national security mission. We

selected all six organizations based on our analysis of documents and

internet searches for organizations that received positive assessments for

their use or employment of AI, and the importance of AI to the

organization. Although our sample is non-generalizable, the information

obtained from officials from the six selected organizations offered useful

H.R. Rep. No. 117-118, at 66 (2021).

For the purposes of this report, the military services include the Army, Marine Corps,

Navy, Air Force, and Space Force, which are a part of DOD during peacetime.

For DOD organizations, we selected a DOD agency, a science and technology

reinvention laboratory, and a federally funded research and development

center/university-affiliated research center. For Defense Intelligence Enterprise

organizations, we selected one military service intelligence organization and two DOD-

level intelligence organizations. Specifically, we selected the Defense Advanced Research

Projects Agency, U.S. Army Combat Capabilities Development Command Army Research

Laboratory, Department of the Air Force-Massachusetts Institute of Technology AI

Accelerator, National Security Agency, National Geospatial-Intelligence Agency, and the

intelligence element of the Air Force.

Appendix I: Objectives, Scope, and

Methodology

Page 36 GAO-24-105645 Artificial Intelligence

insights and perspectives on AI workforce issues in the DOD and DOD’s

efforts to address them.

For objective one, we collected information on the tools human resource

professionals use to recruit AI personnel throughout DOD. We

interviewed and collected written responses to a set of questions from

officials representing six DOD organizations and five military services

about the tools used by DOD to attract AI talent. In addition, we reviewed

relevant policy documents, laws, and regulations that address the tools

used by the department.

For objective two, we relied on documentation and interviews to assess

the steps taken by DOD to define and identify its AI workforce and the

additional steps DOD plans to complete for this process. We compared

those completed and planned steps to the strategic focus areas identified

in the 2018 DOD AI Strategy and requirements established in DOD

regulations. To conduct our assessment, we reviewed DOD documents

including strategies, plans, and regulatory guidance.

Additionally, we

discussed the steps DOD has taken and plans to complete with relevant

DOD officials.

For objective three, we compared the plans and strategies that

incorporate DOD’s AI workforce to the requirements established in federal

regulation and OPM guidance. To conduct our assessment, we reviewed

current and previously published versions of the Human Capital

Operating Plan, Agency Strategic Plan, and Annual Performance Plan.

Additionally, we discussed departmental human capital plans and

strategies, including those in development such as the DOD AI Strategic

Human Capital Plan, with relevant officials. We also discussed service-

level human capital plans and strategies, including those in development,

with officials from the military services. We previously identified strategic

human capital management as a high-risk area for the federal

government.

Documents we assessed include: 2018 DOD AI Strategy; DOD AI Education Strategy

(Sept. 2020); DOD Instruction 1400.25, Volume 250, DOD Civilian Personnel

Management System: Civilian Strategic Human Capital Planning (SHCP) (June 7, 2016);

DOD Directive 5105.79, DOD Senior Governance Framework (Nov. 8, 2021); DOD

Directive 8140.01, Cyberspace Workforce Management (Oct. 5, 2020); and OPM, The

Classifier’s Handbook (August 1991).

GAO, High-Risk Series: Efforts Made to Achieve Progress Need to Be Maintained and

Expanded to Fully Address All Areas, GAO-23-106203 (Washington, D.C.: Apr. 20, 2023).

Appendix I: Objectives, Scope, and

Methodology

Page 37 GAO-24-105645 Artificial Intelligence

To address all our objectives, we interviewed officials and, where

appropriate, obtained documentation from the following organizations:

· DOD

· Chief Digital and AI Office

· Office of the Under Secretary of Defense for Personnel and

Readiness

· Defense Civilian Personnel Advisory Service

· Chief Information Officer

· Office of the Under Secretary of Defense for Research and

Engineering

· Office of the Director of Administration and Management

· DOD Organizations

· U.S. Army Combat Capabilities Development Command Army

Research Laboratory

· Defense Advanced Research Projects Agency

· Department of the Air Force—Massachusetts Institute of

Technology AI Accelerator

· Defense Intelligence Enterprise Organizations

· National Geospatial-Intelligence Agency

· National Security Agency

· Air Force intelligence element

· Military Services

· Army

· Air Force

· Navy

· Marine Corps

· Space Force

· OPM

· Employee Services

We conducted this performance audit from January 2022 to December

2023 in accordance with generally accepted government auditing

standards. Those standards require that we plan and perform the audit to

Appendix I: Objectives, Scope, and

Methodology

Page 38 GAO-24-105645 Artificial Intelligence

obtain sufficient, appropriate evidence to provide a reasonable basis for

our findings and conclusions based on our audit objectives. We believe

that the evidence obtained provides a reasonable basis for our findings

and conclusions based on our audit objectives.

Appendix II: Department of Defense Cyber

Workforce Framework

Page 39 GAO-24-105645 Artificial Intelligence

AppendixII:Departmentof

DefenseCyberWorkforce

Framework

The Office of the Department of Defense (DOD) Chief Information Officer

implemented the DOD Cyber Workforce Framework to meet the

requirements described in the Federal Cybersecurity Workforce

Assessment Act of 2015.

DOD derived its framework from the National

Initiative for Cybersecurity Education Cybersecurity Workforce

Framework (NICE Framework), which was created by the National

Institute of Standards and Technology.

According to DOD, the DOD

Cyber Workforce Framework

· establishes the department’s authoritative terminology based on the

work an individual is performing and not their position title,

occupational series, or designator;

· facilitates the uniform identification, tracking, and reporting required by

the Federal Cybersecurity Workforce Assessment Act of 2015;

· serves as DOD’s cyberspace coding structure for authoritative

personnel and workforce data systems;

and

Pub. L. No. 114-113, § 302 (2015) and codified at 5 U.S.C. § 301 note.

The NICE Framework provides a set of building blocks for describing the tasks,

knowledge, and skills that are needed to perform cybersecurity work performed by

individuals and teams. National Institute of Standards and Technology, “NICE Framework

Resource Center,” accessed Mar. 20, 2023, https://www.nist.gov/itl/applied-

cybersecurity/nice/nice-framework-resource-center.

In December 2022 we recommended that the Army, Marine Corps, and Air Force take

steps to integrate U.S. Cyber Command work roles—which are not contained in the DOD

Cyber Workforce Framework—into service-specific personnel data systems of record to

improve the tracking of cyber personnel. GAO, Military Cyber Personnel: Opportunities

Exist to Improve Service Obligation Guidance and Data Tracking, GAO-23-105423

(Washington, D.C.: Dec. 21, 2022).

The DOD Dictionary of Military and Associated Terms (May 2023) defines cyberspace as

a global domain within the information environment consisting of the interdependent

networks of information technology infrastructures and resident data, including the

internet, telecommunications networks, computer systems, and embedded processors

and controllers.

Appendix II: Department of Defense Cyber

Workforce Framework

Page 40 GAO-24-105645 Artificial Intelligence

· helps organizations recruit, train, educate, and retain a qualified

cyberspace workforce.

In addition to describing the work performed by the full spectrum of the

cyber workforce, the DOD Cyber Workforce Framework has expanded to

include five artificial intelligence (AI) work roles as well as six data and

eight software engineering work roles.

Work roles are used as additional

occupational descriptors, along with civilian occupational series, military

occupational codes, and specialty codes. Each work role contains a

definition; tasks; and knowledge, skills, and ability statements that

describe what is needed to execute critical functions.

For example, the AI/Machine Learning Specialist work role designs,

develops, and modifies AI applications, tools, and other solutions to

enable successful accomplishment of mission objectives. The AI/Machine

Learning Specialist work role includes 21 tasks and 50 knowledge, skills,

and ability statements. See table 2 for examples of those tasks and

knowledge, skills, and ability statements.

Table 2: Examples of Tasks and Knowledge, Skills, and Ability Statements for the Artificial Intelligence (AI)/Machine Learning

Specialist Work Role

Task

Design and develop machine learning models to achieve organizational objectives.

Task

Determine methods and metrics for quantitative and qualitative measurement of AI risks so that sensitivity,

specificity, likelihood, confidence levels, and other metrics are identified, documented, and applied.

Knowledge

Knowledge of emerging trends and future use cases of AI.

Knowledge

Knowledge of how AI is developed and operated.

Skill

Skill in creating machine learning models.

Skill

Skill in testing and evaluating machine learning algorithms or AI solutions.

Ability

Ability to collect, verify, and validate test data.

Source: GAO analysis of Department of Defense (DOD) information. | GAO-24-105645

DOD Instruction 8140.02, Identification, Tracking, and Reporting of Cyberspace

Workforce Requirements (Dec. 21, 2021).

In September 2022 DOD added five AI work roles to the DOD Cyber Workforce

Framework: AI/Machine Learning Specialist (623), AI Test and Evaluation Specialist (672),

AI Risk and Ethics Specialist (733), AI Adoption Specialist (753), and AI Innovation Leader

(902). DOD, “DOD Cyber Workforce Framework,” accessed Mar. 7, 2023,

https://public.cyber.mil/wid/dcwf/.

DOD Instruction 8140.02, Identification, Tracking, and Reporting of Cyberspace

Workforce Requirements and DOD, “DOD Cyber Workforce Framework.”

DOD, “AI/ML Specialist,” DOD Cyber Workforce Framework, accessed Mar. 7, 2023,

https://public.cyber.mil/dcwf-work-role/ai-ml-specialist/.

Appendix II: Department of Defense Cyber

Workforce Framework

Page 41 GAO-24-105645 Artificial Intelligence

The DOD Cyber Workforce Framework leverages the original NICE

Framework.

The concept for the NICE Framework began in 2007 and

grew out of the recognition that the cybersecurity workforce had not been

defined and assessed.

The National Institute of Standards and

Technology posted the first version of the NICE Framework in September

2012.

The NICE Framework provides a set of building blocks for describing the

tasks, knowledge, and skills needed to perform cybersecurity work

performed by individuals and teams. Through these building blocks, the

NICE Framework enables organizations to develop their workforces to

perform cybersecurity work.

Additionally, the NICE Framework helps

provide a common language of the cybersecurity work and of the

individuals who carry out that work. The use of common terms and

language helps to organize and communicate the work to be done and

the attributes of those qualified to perform that work.

DOD, “DOD Cyber Workforce Framework,” accessed Mar. 10, 2023,

https://dodcio.defense.gov/Cyber-Workforce/DCWF/.

The National Institute of Standards and Technology led the development of the NICE

Framework with support from various agencies, to include DOD and the Department of

Homeland Security.

National Institute of Standards and Technology, NICE Framework Resource Center,

accessed Mar. 7, 2023, https://www.nist.gov/itl/applied-cybersecurity/nice/nice-framework-

resource-center and National Institute of Standards and Technology, “NICE Framework

History,” NICE Framework Resource Center (Feb. 28, 2023), accessed Mar. 7, 2023,

https://www.nist.gov/itl/applied-cybersecurity/nice/nice-framework-resource-center/nice-

framework-history.

National Institute of Standards and Technology Special Publication 800-181, revision 1,

Workforce Framework for Cybersecurity (NICE Framework) (Nov. 2020).

National Institute of Standards and Technology, Workforce Framework for

Cybersecurity.

Appendix III: Comments from the Department

of Defense

Page 42 GAO-24-105645 Artificial Intelligence

AppendixIII:Commentsfromthe

DepartmentofDefense

Appendix III: Comments from the Department

of Defense

Page 43 GAO-24-105645 Artificial Intelligence

Appendix III: Comments from the Department

of Defense

Page 44 GAO-24-105645 Artificial Intelligence

Appendix III: Comments from the Department

of Defense

Page 45 GAO-24-105645 Artificial Intelligence

TextofAppendixIII:CommentsfromtheDepartment

ofDefense

Ms. Alissa H. Czyz

Director, Defense Capabilities and Management

U.S. Government Accountability Office 441 G Street, NW

Washington DC 20548 Dear Ms. Czyz,

Enclosed is the Department of Defense (DoD) proposed response to the GAO Draft

Report GAO-24-105645SU "ARTIFICAL INTELLIGENCE: Actions Needed to

Improve DOD's Workforce Management," dated August 31, 2023 (GAO Code

105645).

Should you have any questions, please contact Ms. Sukeena Blair, Primary Action

Officer, at [email protected], or (571) 372-1666.

Sincerely,

Grier Martin

Performing the Duties of the Assistant Secretary of Defense for Manpower and

Reserve Affairs

Enclosure:

As stated

GAODRAFTREPORTDATEDAUGUST31,2023,GAO24

105645SU(GAOCODE105645),“ARTIFICIALINTELLIGENCE:

ActionsNeededtoImproveDOD’sWorkforceManagement”

DEPARTMENT OF DEFENSE COMMENTS TO THE GAO

RECOMMENDATION

RECOMMENDATION 1: The Secretary of Defense should ensure the CDAO

assigns responsibility to complete the additional steps necessary to fully

define and identify the AI workforce, to include coding the work rules in

various personnel data systems, developing an AI competency model,

Appendix III: Comments from the Department

of Defense

Page 46 GAO-24-105645 Artificial Intelligence

determining the scope of the AI workforce, and other steps identified by the

CDAO.

DoD RESPONSE: DoD Partially Concurs. As written, there is a conflation of

workforce definition and identification processes for work roles compared to

occupation series. There are also inaccuracies regarding the order of operations.

DoD proposes revising this recommendation to read: “The Secretary of Defense

should ensure that the CDAO completes the additional steps necessary to fully

define and identify the AI workforce, to include publishing interpretive guidance,

setting work role qualifications, assigning components to code the work roles in

various manpower data systems and personnel data systems, and other steps

identified by the CDAO in coordination with the Under Secretary of Defense for

Personnel and Readiness.

RECOMMENDATION 2: The Secretary of Defense should ensure that the

CDAO establishes a timeline for additional steps necessary to fully define and

identify the AI workforce, to include coding the work roles in various personnel

data systems, developing an AI competency model, and determining the scope

of the AI workforce, and other steps identified by the CDAO.

DoD RESPONSE: DoD Partially Concurs. As written, there is a conflation of

workforce definition and identification processes for work roles compared to

occupation series. There are also inaccuracies regarding the order of operations.

DoD proposes revising this recommendation to read: “The Secretary of Defense

should ensure that the CDAO establishes a timeline for additional steps necessary to

fully define and identify the AI workforce, to include publishing interpretive guidance,

setting work role qualifications, assigning components to code the work roles in

various manpower data systems and personnel data systems, and other steps

identified by the CDAO in coordination with the Under Secretary of Defense for

Personnel and Readiness.”

RECOMMENDATION 3: The Secretary of Defense should ensure that the

Under Secretary of Defense for Personnel and Readiness updates the Human

Capital Operating Plan to be consistent with the Agency Strategic Plan and

Annual Performance Plan – as it relates too AI workforce issues – in the next

annual review. This should include: (1) addressing the human capital

implementation actions planned to support the strategic goals and priorities

identified in the Agency Strategic Plan and Annual Performance Plan, and (2)

ensuring the use of consistent AI terminology.

DoD RESPONSE: DoD Partially Concurs. The title 5, Code of Federal Regulations

250, Subpart B, §250.204 “Agency roles and responsibilities” states at (a)(1) “Align

with Executive branch policies, and priorities, as well as with individual agency

Appendix III: Comments from the Department

of Defense

Page 47 GAO-24-105645 Artificial Intelligence

missions, goals, and strategic objectives. Agencies must align their human capital

management strategies to support the Federal Workforce Priorities Report, agency

strategic plan, agency performance plan, and agency budget.” In accordance with

§250.204(a)(1), DoD has recently completed a Human Capital Operating Plan

Refresh for Fiscal Year (FY) 2022 and FY 2026. This refresh includes strategic

alignment to the President Management Agenda, Federal Workforce Priorities

Report, the DoD Strategic Management Plan, and other related strategic initiatives.

As such, DoD proposes revising this recommendation to read: “The Secretary of

Defense should ensure that the Under Secretary of Defense for Personnel and

Readiness updates the Human Capital Operating Plan to be consistent with the

Agency Strategic Plan and Annual Performance Plan - engaging all stakeholders, as

appropriate, relating to AI workforce issues in the next annual review. This should

include: (1) addressing the human capital implementation actions planned to support

the strategic goals and priorities identified in the Agency Strategic Plan and Annual

Performance Plan, and (2) ensuring the use of consistent AI terminology.”

Appendix IV: GAO Contact and Staff

Acknowledgments

Page 48 GAO-24-105645 Artificial Intelligence

AppendixIV:GAOContactand

StaffAcknowledgments

GAOContact

Alissa H. Czyz, (202) 512-3058 or [email protected].

StaffAcknowledgments

In addition to the contact named above, Penney Harwell Caramia

(Assistant Director), Andrew Avery, Tracy Barnes, Pamela Davidson, Erik

Goepner, Paris Hall, David Hinchman, Amie Lesser, Steven Lozano,

Lillian Ofili, Lucas Smith, Jessica Steele, and Emily Wilson made key

contributions to this report.

GAO’sMission

The Government Accountability Office, the audit, evaluation, and investigative

arm of Congress, exists to support Congress in meeting its constitutional

responsibilities and to help improve the performance and accountability of the

federal government for the American people. GAO examines the use of public

funds; evaluates federal programs and policies; and provides analyses,

recommendations, and other assistance to help Congress make informed

oversight, policy, and funding decisions. GAO’s commitment to good government

is reflected in its core values of accountability, integrity, and reliability.

ObtainingCopiesofGAOReportsandTestimony

The fastest and easiest way to obtain copies of GAO documents at no cost is

through our website. Each weekday afternoon, GAO posts on its website newly

released reports, testimony, and correspondence. You can also subscribe to

GAO’s email updates to receive notification of newly posted products.

OrderbyPhone

The price of each GAO publication reflects GAO’s actual cost of production and

distribution and depends on the number of pages in the publication and whether

the publication is printed in color or black and white. Pricing and ordering

information is posted on GAO’s website, https://www.gao.gov/ordering.htm.

Place orders by calling (202) 512-6000, toll free (866) 801-7077, or

TDD (202) 512-2537.

Orders may be paid for using American Express, Discover Card, MasterCard,

Visa, check, or money order. Call for additional information.

ConnectwithGAO

Connect with GAO on Facebook, Flickr, Twitter, and YouTube.

Subscribe to our RSS Feeds or Email Updates. Listen to our Podcasts.

Visit GAO on the web at https://www.gao.gov.

ToReportFraud,Waste,andAbuseinFederal

Programs

Contact FraudNet:

Website: https://www.gao.gov/about/what-gao-does/fraudnet

Automated answering system: (800) 424-5454 or (202) 512-7700

CongressionalRelations

A. Nicole Clowers, Managing Director, [email protected], (202) 512-4400, U.S.

Government Accountability Office, 441 G Street NW, Room 7125, Washington,

DC 20548

PublicAffairs

Chuck Young, Managing Director, [email protected], (202) 512-4800

U.S. Government Accountability Office, 441 G Street NW, Room 7149

Washington, DC 20548

StrategicPlanningandExternalLiaison

Stephen J. Sanford, Managing Director, [email protected], (202) 512-4707

U.S. Government Accountability Office, 441 G Street NW, Room 7814,

Washington, DC 20548