When we have a question about something embarrassing or deeply personal, many of us don't turn to a parent or a friend, but to our computers: We ask Google our questions.
As millions of us look for answers to questions, or things to buy, or places to meet friends, our searches produce a map of our collective hopes, fears, and desires.
Seth Stephens-Davidowitz, a former data scientist at Google, analyzes the information we leave behind on search engines, social media, and even pornography sites. He's the author of the book Everybody Lies: Big Data, New Data, and What the Internet Can Tell Us About Who We Really Are.
"I think there's something very comforting about that little white box that people feel very comfortable telling things that they may not tell anybody else about: Their sexual interests, their health problems, their insecurities. And using this anonymous aggregate data we can learn a lot more about people than we've really ever known," he said.
By mining data from the Internet, Stephens-Davidowitz has found surprising correlations that tell a far different story than those presented by surveys. Online data allow him, for example, to estimate the percentage of American men who are gay; predict the unemployment rate weeks before the federal government releases official statistics; and uncover parents' unconscious biases against girls.
It's not just researchers like Stephens-Davidowitz who have figured out big data's ability to reveal truth. Companies already use big data to predict our behavior — from whether we'll pay back a loan to which movies we'll watch on the weekend.
These new forms of data are so valuable, Stephens-Davidowitz argues, because they understand us better than we understand ourselves.
Hidden Brain is hosted by Shankar Vedantam and produced by Maggie Penman, Jennifer Schmidt, Rhaina Cohen, and Renee Klahr. Our supervising producer is Tara Boyle. You can follow us on Twitter @hiddenbrain, and listen for Hidden Brain stories each week on your local public radio station.
Please take our brief audience survey.
You won’t find a paywall here. Come as often as you like — we’re not counting. You’ve found a like-minded tribe that cherishes what a free press stands for. If you can spend another couple of minutes making a pledge of as little as $5, you’ll feel like a superhero defending democracy for less than the cost of a month of Netflix.