Philip Bump wrote a really interesting article on The Atlantic Wire today on how big (really) the data storage required for the NSA‘s now infamous PRISM ‘spying’ project would be.
Bump’s calculations are dependant on this hunk of hardware: the HP ProLiant DL580 G7 — currently the ‘highest-density system’ commercially available. You can read the exact calculations in his article, but he puts the estimate storage space of these behemoths at 21 terabytes. Here’s a size and storage comparison with Blu Ray’s and DVD’s:
Moving forward, Bump proposes the NSA would need 5,600 boxes to store a single year’s worth of PRISM data (9.7-petabytes: a modest 406,847 25GB Blu Rays). As a data visualisation exercise, this is where things get really interesting: that’s a huge amount of boxes. How big is that in comparison to other “big” things?
Let’s start with the simplest comparison: let’s forget gravity and a host of other physical restrictions and stack these boxes one on top of each other. 5,600 x 17.6 centimetres (6.94 “) is a huge 861 metres (around 2,825 feet). Compare that below to some of the world’s tallest and iconic buildings:
That’s an impressive tower of data. But not particularly practical. Bump mentions stacking the boxes:
If you stack those boxes 5 feet high (eight boxes), we’re talking about a total physical footprint of 2,410 square feet.
2,410 square feet — or (223.9 square metres) — is another big big number. But let’s forget stacking them and consider how much space you’d need if they were laid flat (no stacking). A single HP ProLiant DL580 has an area of 3.63 square feet (0.338 square metres). Now for a year, based on 5,600 boxes, that gives us 20,380 square feet (1,893.36 square metres). That’s around 26 soccer pitches.
Let’s compare it to some actual landmarks, overlaying the area required for PRISM layed flat:
New York / Times Square
London / Trafalgar Square
Paris / The Louvre
Now we’ve laid the drives into a square shape, they are far more compact: which shows that “stacking” them up in the air gives quite a skewed representation of how much space we’re actually talking.
This lot of data in a not lot of space — of course with zero provision for cooling systems, electronics etc. So not too worry, the NSA has plenty of space: this year in Utah, the NSA will complete a new data centre with an area of 1 million square feet (92,903 square metres).
Rest assured, the NSA will be stacking these servers — and will have PLENTY of space for operations like PRISM.
(I hope these calculations all work — any errors please let me know!)
Update: (12 June) I made some errors sorting out my area calculations that I’ve updated above. Thanks very much to Dr. Bob Campbell for helping me correct this!