Transparency Tip: Scouring Public Records with Scout

If you do any kind of shopping online, then you’re probably already familiar with the concept of a metasearch engine—a search engine that searches other search engines. When you’re looking for airline tickets or a used copy of The King in Yellow, you might use a metasearch engine to find out which online retailer is offering the best price.

There is also a metasearch for public records: Scout. Created and lovingly maintained by the Sunlight Foundation, Scout will take a search query and run it through six different continually updated databases of documents. The site also allows you to create email and RSS alerts and track search terms and legislation through a user account.

EFF currently maintains a public page that tracks NSA reform, including congressional bills and speeches, official notices, and court opinions. You can visit it to see current reform efforts and subscribe to our alerts.

How to use Scout

Scout currently searches six datasets: federal legislation bills, court opinions (via courtlistener.com), federal regulations, state bills, oversight body reports, and Congressional floor speeches. The best place to start is with Sunlight Foundation’s video tutorial by Eric Mill.

Privacy info. This embed will serve content from youtube-nocookie.com

The short version: Just type in a word and hit search, or check “Advanced” and enter a phrase with quotes. If you want regular updates, click “Create Alert” on any given page. Go to “Manage Alerts” to set Scout to email you immediately or daily, or to send you a text message. That’s also where you can grab the RSS feed.

Simple as that.

Scout’s strength is its speed, convenience, and the way it helps researchers discover relevant documents that they may not have known existed.

For EFF, Scout is handy for discovering and tracking bills that would impact our rights. “FISA” (Foreign Intelligence Surveillance Act) is a particularly fruitful search term for following NSA reform, while “DMCA” (Digital Millennium Copyright Act) will return results relevant to changes to copyright law. It’s helpful to compare these results to what legislators are saying about the bills, which you can find if you also look at the results for “Speeches in Congress.” Scout allows you to sift through those speeches by state, party, chamber, and individual lawmaker. (Here’s a solid anti-NSA screed from Sen. Patrick Leahy.)

Scout will also return results from state legislation, which would be difficult to track if you had to go to each state’s individual websites. Since state legislators have the authority to regulate commerce and set law enforcement policies, they will often take up electronic privacy issues, such as drones, automatic license plate readers, and consumer data mining. You can also get a good sense of how the states feel about federal issues—for example, Alaska, Virginia, California, Pennsylvania, Michigan, and Ohio all have legislation or resolutions opposing mass surveillance by the NSA.

However, because data standards vary across the country, state legislation isn’t quite as easy for Scout to process as federal legislation. If you identify a bill worth deeper inspection, you should take the bill number from Scout and plug it directly into the individual state legislature’s web site.

A substantial amount of US policy is determined through the federal regulatory process, which is why you’ll also want to review the results Scout finds in the federal register. These can range from notices from bodies such as the Privacy and Civil Liberties Oversight Board to a listing of all the gifts (e.g., luxury wrist watches, rugs, vases, ceremonial blades, firearms) the CIA chief accepted from anonymous foreign governments because “non-acceptance would cause embarrassment to donor and U.S. Government.”

And, of course, the oversight reports—obtained from the Government Accountability Office and various inspectors general—are often filled with important nuggets about questionable government programs. These reports are also good for grabbing statistics and background details on particular issues. Scout only displays a summary of the reports, so you’ll need to follow the PDF link to read each in full.

One last tip: As all-powerful as Scout seems, it’s not infallible and that’s one reason why it always links back to the original material. If in doubt, or if you’re conducting a fact check, always refer back to the source.

Other resources:

USA.gov –  A not-particularly-reliable search engine that is suppose to return results from across government networks. If there is a specific document you’re after, you’re not likely to find it here. It’s like dropping a net into an unorganized ocean of information; you could very well find something meaty you didn’t know you were looking for or you could find nothing but old boots and plankton. On the other hand, if you’re looking for public domain images of anything related to government, USA.gov’s image search is a great resource.

Google Advanced Search – Here’s a little hack for Google that will sometimes return different results than USA.gov. Put in your search term as you normally would, but add in “site:.gov” right after it (also try site:.mil for military sites).

With both USA.gov and Google, you can add in “filetype:pdf”, “filetype:ppt” or “filetype:xls” to your search to narrow results to only include PDF, PowerPoint or Excel files.

You can find all of EFF’s Sunshine Week posts linked here.