Data Warehouse System Electronic Surveillance Data Management System


The Data Warehouse System — Electronic Surveillance Data Management System is an electronic database created by the Special Technologies and Applications Section of the Federal Bureau of Investigation. Disclosed in a heavily redacted review of the FBI's role in the prevention of the 2009 Fort Hood shooting, its full capabilities are classified but at a minimum, provides a searchable archive of intercepted electronic communications, including email sent over the Internet. Another report suggests that online chat transcripts, email attachments, and audio of unspecified origin are stored.
By the third quarter of 2006 it was involved in 130 successful investigations and 370 active cases. In June 2007 a total of 70 million intercepts from 16,500 online accounts were present on the system, and were expected to increase to 350 million intercepts from over 50,000 accounts by June 2009.

History

The DWS was designed by STAS in 2001 to record a certain class of data intercepts. In the following years it became a de facto depository of a wider category of intercepts, surpassing its original intent and straining its hardware limits.
The DWS was involved in the interception of 18 email messages between Nidal Malik Hasan and Anwar al-Awlaki from December 2008 to June 2009, although the messages were largely benign and provided no actionable evidence.
It was upgraded in February 2009, improving the user interface and tools, and merged with a related system, EDMS. The user interface was again improved in May 2009.

Capabilities

The interface of DWS-EDMS is described as relatively crude. Intercepts are stored in text format. The primary user interface is similar to Outlook Express. Users can add notes, translations, and tags to intercepts. While it provides search capabilities, it does so poorly, failing to return many relevant results. Although it is one of the FBI's primary investigative tools, it does not have a backup scheme in place. A failure scenario such as data corruption would be considered catastrophic. As of 2011, it was considered to be overburdened and incapable of sustaining its present role.
The underlying infrastructure of DWS-EDMS uses widely available commercial and open source technologies, including Java, Red Hat Linux, Oracle SQL, XML, Microsoft IIS, and Apache HTTP Server.