WASHINGTON — For more than five years, American intelligence agencies followed several groups of Chinese hackers who were systematically draining information from defense contractors, energy firms and electronics makers, their targets shifting to fit Beijing’s latest economic priorities.
But last summer, officials lost the trail as some of the hackers changed focus again, burrowing deep into United States government computer systems that contain vast troves of personnel data, according to American officials briefed on a federal investigation into the attack and private security experts.
Undetected for nearly a year, the Chinese intruders executed a sophisticated attack that gave them “administrator privileges” into the computer networks at the Office of Personnel Management, mimicking the credentials of people who run the agency’s systems, two senior administration officials said. The hackers began siphoning out a rush of data after constructing what amounted to an electronic pipeline that led back to China, investigators told Congress last week in classified briefings.
Much of the personnel data had been stored in the lightly protected systems of the Department of the Interior, because it had cheap, available space for digital data storage. The hackers’ ultimate target: the one million or so federal employees and contractors who have filled out a form known as SF-86, which is stored in a different computer bank and details personal, financial and medical histories for anyone seeking a security clearance.
“This was classic espionage, just on a scale we’ve never seen before from a traditional adversary,” one senior administration official said. “And it’s not a satisfactory answer to say, ‘We found it and stopped it,’ when we should have seen it coming years ago.”
The administration is urgently working to determine what other agencies are storing similarly sensitive information with weak protections. Officials would not identify their top concerns, but an audit issued early last year, before the Chinese attacks, harshly criticized lax security at the Internal Revenue Service, the Nuclear Regulatory Commission, the Energy Department, the Securities and Exchange Commission — and the Department of Homeland Security, which has responsibility for securing the nation’s critical networks.
At the Nuclear Regulatory Commission, which regulates nuclear facilities, information about crucial components was left on unsecured network drives, and the agency lost track of laptops with critical data.
Computers at the I.R.S. allowed employees to use weak passwords like “password.” One report detailed 7,329 “potential vulnerabilities” because software patches had not been installed. Auditors at the Department of Education, which stores information from millions of student loan applicants, were able to connect “rogue” computers and hardware to the network without being noticed. And at the Securities and Exchange Commission, part of the network had no firewall or intrusion protection for months.
“We are not where we need to be in terms of federal cybersecurity,” said Lisa Monaco, President Obama’s homeland security adviser. At an Aspen Institute conference in Washington on Tuesday, she blamed out-of-date “legacy systems” that have not been updated for a modern, networked world where remote access is routine. The systems are not continuously monitored to know who is online, and what kind of data they are shipping out.
In congressional testimony and in interviews, officials investigating the breach at the personnel office have struggled to explain why the defenses were so poor for so long. Last week, the office’s director, Katherine Archuleta, stumbled through a two-hour congressional hearing. She was unable to say why the agency did not follow through on inspector general reports, dating back to 2010, that found severe security lapses and recommended shutting down systems with security clearance data.
When she failed to explain why much of the information in the system was not encrypted — something that is standard today on iPhones, for example — Representative Stephen F. Lynch, a Massachusetts Democrat who usually supports Mr. Obama’s initiatives, snapped at her. “I wish that you were as strenuous and hardworking at keeping information out of the hands of hackers,” he said, “as you are keeping information out of the hands of Congress and federal employees.”
Her performance in classified briefings also frustrated several lawmakers. “I don’t get the sense at all they understand the problem,” said Representative Jim Langevin, a Rhode Island Democrat, who called for Ms. Archuleta’s resignation. “They seem like deer in the headlights.”
Josh Earnest, the White House spokesman, said on Wednesday that Mr. Obama remained confident that Ms. Archuleta “is the right person for the job.” Ms. Archuleta, who took office in November 2013, did not respond to a request for an interview.
But even some White House aides say a lack of focus by managers contributed to the security problems. It was not until early last year, as computer attacks began on United States Investigations Services, a private contractor that conducts security clearance interviews for the personnel office, that serious efforts to develop a strategic plan to seal up the agency’s many vulnerabilities started.
The attacks on the contractor “should have been a huge red flag,” said one senior military official who has reviewed the evidence of China’s involvement. “But it didn’t set off the alarms it should have.”
Federal and private investigators piecing together the attacks now say they believe the same groups responsible for the attacks on the personnel office and the contractor had previously intruded on computer networks at health insurance companies, notably Anthem Inc. and Premera Blue Cross.
What those attacks had in common was the theft of millions of pieces of valuable personal data — including Social Security numbers — that have never shown up on black markets, where such information can fetch a high price. That could be an indicator of state sponsorship, according to James A. Lewis, a cybersecurity expert at the Center for Strategic and International Studies.
But federal investigators, who like other officials would not speak on the record about a continuing inquiry, said the exact affiliation between the hackers and the Chinese government was not fully understood. Their tools and techniques, though, were easily identifiable to intelligence analysts and the security researchers who have been analyzing the breaches at the insurers and the Office of Personnel Management. Federal officials believe several groups were involved, though some security experts only detected one.
“Since mid-2014, we have observed a threat group target valuable ‘personally identifiable information’ from multiple organizations in the health care insurance and travel industries,” said Mike Oppenheim, the manager of threat intelligence at FireEye, a cybersecurity company. “We believe this group is behind the O.P.M. breach and have tracked this group’s activities since early 2013.”
But he argued that “unlike other actors operating from China who conduct industrial espionage, take intellectual property or steal defense technology, this group has primarily targeted information that would enable it to build a database of Americans, with a likely focus on diplomats, intelligence operatives and those with business in China.”
While Mr. Obama publicly named North Korea as the country that attacked Sony Pictures Entertainment last year, he and his aides have described the Chinese hackers in the government records case only to members of Congress in classified hearings. Blaming the Chinese in public could affect cooperation on limiting the Iranian nuclear program and tensions with China’s Asian neighbors. But the subject is bound to come up this week when senior Chinese officials meet in Washington for an annual strategic and economic dialogue.
Though their targets have changed over time, the hackers’ digital fingerprints stayed much the same. That allowed analysts at the National Security Agency and the F.B.I. to periodically catch glimpses of their movements as they breached an ever more diverse array of computer networks.
Yet there is no indication that the personnel office realized that it had become a Chinese target for almost a year. Donna K. Seymour, the chief information officer, said the agency put together last year “a very progressive, proactive plan that allowed us to see the adversarial activity,” and argued that “had we not been on that path, we may never have seen anything” this spring. She cautioned, “There is no one security tool that is a panacea.”
A congressional report issued in February 2014 by the Republican staff of the Senate Homeland Security Committee, concluded that multiple federal agencies with responsibility for critical infrastructure and holding vast amounts of information “continue to leave themselves vulnerable, often by failing to take the most basic steps towards securing their systems and information.”
The report reserves its harshest criticism for the repeated failures of agency officials to take steps — some of them very basic — that would help thwart cyberattacks.
Computers at the Department of Homeland Security, which is charged with protecting the nation’s public infrastructure, contained hundreds of vulnerabilities as recently as 2010, according to authors of the report. They said computer security failures remained across agencies even though the government has spent “at least $65 billion” since 2006 on protective measures.
At the personnel office, a set of new intrusion tools used on the system set off an alarm in March, Ms. Seymour said. The F.B.I. and the United States Computer Emergency Response Team, which works on network intrusions, found evidence that the hackers had obtained the credentials used by people who run the computer systems. Ms. Seymour would say only that the hackers got “privileged user access.” The administration is still trying to determine how many of the SF-86 national security forms — which include information that could be useful for anyone seeking to identify or recruit an American intelligence agent, nuclear weapons engineer or vulnerable diplomat — had been stolen.
“They are casting a very wide net,” John Hultquist, a senior manager of cyberespionage threat intelligence at iSight Partners, said of the hackers targeting of Americans’ personal data. “We’re in a new space here and we don’t entirely know what they’re trying to do with it.”