• Question: Is there ever 'too' much data for you to handle?

    Asked by Tiffany5567 to Anna, George, Jodi, Rob, Stefan on 18 Nov 2014.
    • Photo: Anna Scaife

      Anna Scaife answered on 18 Nov 2014:

      At the moment, yes, definitely. That’s why it’s so important to improve our methods of storing and processing data, because the datasets are only ever going to get bigger… 🙂

    • Photo: Jodi Schneider

      Jodi Schneider answered on 18 Nov 2014:

      When I first started researching Wikipedia, I downloaded the “data dumps” that have the whole encyclopedia. But I found out that they were too big to unpack & work on in my lab. I had to find other ways to look at “slices” of the data. So yes!!!

    • Photo: Stefan Lines

      Stefan Lines answered on 18 Nov 2014:

      Yes! What is difficult is storing the data for analysis. Particularly, the super computer generates a lot of data, but cannot store it all. So I have to move it around… before the disk space fills up!

Comments