Federal agencies are constantly seeking innovative, secure solutions to improve their operations and better serve constituents. One of the technologies that has caught their attention in the past decade is artificial intelligence.

With the recent push of Large Language Model and Transformer AI, the federal workforce has become enthralled with the potential of AI and how it can empower them to effectively carry out agency missions.

One popular LLM tool is ChatGPT. Lawmakers are already using ChatGPT to help write speeches, and agencies have begun to investigate the countless other benefits of implementing the technology within existing processes, including assisting employees in communicating, streamlining workflows, and increasing employees’ access to information.

Amid the excitement, there are concerns that ChatGPT and other LLM tools could eliminate jobs, provide inaccurate information and perpetuate bias. While some fears may be unfounded, it’s crucial that federal agencies consider all the potential impacts a full-scale implementation of ChatGPT may have on the workforce and agency stakeholders.

To do so requires a thorough understanding of where ChatGPT can help serve the federal workforce and, more importantly, where it can’t.

ChatGPT is powerful, but not all-powerful

Traditionally, AI assists government employees with internal processes by helping at service desks, streamlining decisions, automating repetitive tasks, and more. While ChatGPT can fulfill those traditional roles, its large language model also enables a new dimension of capabilities.

The technology can provide the federal workforce enhanced training and professional development opportunities by creating online courses, tutorials, and other educational resources that federal employees can access anytime. Alternatively, it could improve employee access to information about policies, procedures, regulations, and work-relevant data and statistics.

ChatGPT also has the potential to assist in solving well-documented government-wide challenges. For example, it can help streamline the complex federal acquisition process by drafting a government contract that employees can edit instead of creating from scratch. The potential applications are not lost on agencies, as the Department of Defense is already in the process of creating a similar AI-powered contract-writing solution known as “AcqBot” to accelerate workflows.

What’s more, the applications of ChatGPT can easily be tailored for specific agency needs. The IRS could use ChatGPT to automate the process of categorizing incoming tax forms and routing them to the appropriate department for processing, while DoD could use it to automatically generate reports on equipment maintenance, streamlining the scheduling of repairs.

The applications of ChatGPT are limited only by imagination, creativity, and desired level of impact. However, the effects are not all positive, and relying too much on ChatGPT can bring significant risk to an organization without the proper guardrails. For example, certified ethical hacking community members have proven ways to jailbreak numerous LLM and Transformer-based tools like ChatGPT and trick them into inserting bias into answers, violating ethical policies, creating code for Distributed Denial-of-Service attacks and more.

ChatGPT can also occasionally provide responses or extract information from sources that do not exist to provide a prompt answer. This happens because ChatGPT uses probability-based guessing based on available information to generate its responses.

Therefore, while the original data it pulls from may be accurate, compressing and adjusting the data to answer the unique prompt may generate the most likely statement instead of an accurate one, resulting, at times, in entirely made-up sources. In the event of an error, it won’t acknowledge its mistake but reaffirm its response. This may lead to potential misinterpretations by federal employees relying on the tool to obtain information on benefits or formatting errors in contracts drafted using the tool.

Additionally, if an employee unknowingly uses the information to make a decision that results in a negative response, they may suffer a loss of time, resources, and even reputation. Therefore, it is essential to ensure that ChatGPT is properly tested and monitored, and that employees are trained to minimize the risk of errors before using the system.

Is it ethical to use ChatGPT?

Government agencies are responsible for protecting the safety and well-being of all citizens, including the federal workforce. Many fear that using ChatGPT to assist the workforce will lead to real humans losing their jobs or accidentally perpetuating biases.

These fears are not wholly unfounded, as ChatGPT is incapable of human or critical thought. Because it can’t make emotional decisions or determine when it’s being prejudiced, if the data used to inform its research is biased, the bias may be amplified in the responses provided, which can have profound implications and consequences for government agencies that are expected to provide impartial information to citizens. This means agencies with any citizen-facing interactions must be very careful with how they use the technology when interacting with the public or making decisions that have wide-reaching impacts.

These limitations also provide good news as it means that ChatGPT can’t replace humans. As with all technology, it has tasks in which it excels and tasks that it fails -- these lessons remind us that technology exists to empower, not replace, humans. The ultimate rule of technology holds true for ChatGPT: Don’t trust; always verify. It is human’s ability to question and confirm that AI can never replace.

Integrating ChatGPT into federal systems requires a considerable investment of time, resources, and education to ensure that it performs its tasks and responsibilities effectively without causing harm. Agencies must understand that ChatGPT is not a panacea for all challenges. To unlock its full potential, the government must carefully tailor the tool for specific applications while educating employees on its capabilities, applications and risks.

While more human involvement will be necessary in the short term to ensure that ChatGPT is used effectively and safely, the future benefits of ChatGPT are vast and will undoubtedly lead to a more efficient and effective government for all.

Original Federal Times Article