Tough technological problems can not be solved as side projects

(written by lawrence krubner, however indented passages are often quotes). You can contact lawrence at: lawrence@krubner.com, or follow me on Twitter.

In different ways, I’ve run into this sort of simplistic thinking with many of my clients:

During grad school, I did a lot of consulting for ‘data startups’ (before ‘big data’ was a thing) and consistently ran into the same story: smart founders, usually not technical, have some idea that involves NLP or ML and they come to me to just ‘hammer out a model’ for them as a contractor. I would spend a few hours trying to get concrete about the problem they want to solve and then explain why the NLP they want is incredibly hard and charitably years away from being feasible; even then they’d need a team of good NLP people to make it happen, not me explaining ML to their engineers on the board a few hours a week. Useable fine-grained sentiment analysis is not going to be solved as a side project.

Often the founders of these companies were indeed finding real pain points, but their view of ML was that it was some kind of ‘magic sauce’ they could sprinkle on an idea to make a product. None of the thinking was constrained by what was feasible or likely to even work. They also couldn’t recognize the data problems they had and which ones they could solve, because they weren’t used to viewing the world in that way. Machine learning, if it’s a key part of a product, can’t just be grown and attached to a company’s core unless it’s there from the start and baked into the foundation.

Post external references

  1. 1
    http://cacm.acm.org/blogs/blog-cacm/157645-a-funny-thing-happened-on-the-way-to-academia/fulltext
Source