I started working with computers a long-long time ago, in another galaxy. At that time, limits were always present. Those who have lived with COBOL , 5 1/4-inch floppy disks or the CGA video modes understand what I mean. Limits were everywhere, and hacking our way within these limits was a large part of the fun. We had discipline and skills to work within these limits.
Yesterday, our main limit was time. The time it takes to process content, and the time it takes to develop the programs. The first limit apparently (according to the newspapers) has disappeared for the enterprise world with the emergence of Big Data technologies. Who cares about disk space or RAM anymore? Who cares about data volumes? Big Data lets you process any volume of information in no time.
Throwing the processing equivalent of the Death Star at a problem does not solve it automatically. You need something special to get to the essence of a problem and solve it with a single stroke. This is where Jedis are handy. These are rare and efficient experts. They use super cool light sabers. A Jedi can beat a Death Star.
Similarly, a good data scientist with appropriate information cuts his way through data problems way more efficiently than a galaxy-sized cluster handled by an army and fed with poorly managed data.
The new (and old) discipline that solves today’s problems is simply data management and governance.
The Semarchy Team wishes you a Merry Christmas and a Happy New Year!