With their thousands of software engineers, huge resources and myriad databases, the Googles, Amazons and Facebooks may seem to inhabit an alternative IT universe.
But what the big web services firms are doing today with data volumes and data types will soon be the common experience for many businesses, according to Neo Technology CEO Emil Eifrem…
“If you’re interested in seeing the future of how data-oriented architectures are likely to evolve, the future is already here — just unevenly distributed,” Eifrem said.
“What that means is if you look at some of the big web services — the Googles and the Amazons of the world — they are already today dealing with the volume and shape of data that everyone else will be working on in five years from now…”
And our government – courtesy of their pat 3-letter agencies – is even bigger.
“When you receive an email in your Gmail account, that’s going to be chopped up into various forms and stored as a simple log over here in maybe the equivalent of a document database. But then over here all the contacts and the keywords are going to be stored in a graph database. Then they have really awesome systems for keeping all that data in sync,” he said.
“It’s not perfect and it’s a lot of hard work. But it’s the reality that we’re all going to face and a lot of people are already facing it today.”
Eifrem said companies are already past the point where a single database is capable of managing all data workloads — and it’s misleading for any vendor to suggest it has the answer to all an enterprise’s database problems…
“If you don’t go to the pains of choosing that technology and getting that thousand times improvement, then someone else in your vertical will. They’re just going to build a much better product and glean so much better insights that they’re going to outperform you…
“Every single dataset today or tomorrow is going to be big. So the role of the data architect in the future is going to be to look at my big dataset and then identify parts of it that are shaped as tables and say, ‘That fits really well in my trusted old relational database from Oracle or IBM or whatever,'” Eifrem said.
RTFA and keep this fact in the front of your brain: the dataset being discussed is you.
Eifrem sells products which perform best at slicing and dicing all the information available about your life and how you live — to sell you something and make a profit. That’s not what Uncle Sugar does with the same or similar software.
The government we confront – through Congress, through a compliant Supreme Court, led by whoever is in the White House from the Age of Reagan forward – compiles every speck of information on you, me and the dog to assure themselves that we’re going to stay in line. Or be kept in line. Some geeks will revolt! Some will accept a paycheck – and cooperate
Regardless of how often and how loudly they lie, the two qualities our government fears the most are transparency and democracy.