Streaming Algorithms, Scala eXchange Edition. Or, Stop Analysing and Start Acting
Published on 22 November 2012
Compared to previous talks I spent much more time on motivation. Analytics are only part of the build-measure-learn loop (or, if you prefer, the scientific method) and I wanted to put them in context, and, to be honest, motivate people to look beyond analytics. The focus of the big data community still seems to be on collecting data and performing elementary analyses on it. This misses the point that if your data doesn’t lead to action there is no value in collecting it. Furthermore, optimising your speed through the build-measure-learn loop can be a huge win. The best way to speed up a process is to automate it and, as we’re showing with Myna, this is entirely viable. I truly believe that if data scientists are to realise their true value they need reposition themselves as a stage in the feedback loop, to a critical component overseeing and optimising the entire loop.