Machine Learning Motivations

I've been doing a fair bit of reading on machine learning lately, and I need a dumping ground for my notes, thoughts, and todo lists. A place to think-out-loud. Normally, I'd just stash it all in a text file, but I have a great track record for periodically losing everything on my computer. This blog has proven to be remarkably durable in comparison, and I think my future self may find some of this interesting.

So why machine learning? It's a silly thing, but I recently picked up a retro-style space shoot-em-up game on the cheap. I played it for a few minutes, made a little progress, and thought, "You know, what would really be interesting is trying to get a computer to play this. Certainly more interesting than playing it myself." Back when I first started learning to program, I had had a similar thought about Sudoku, and wrote a fairly simplistic Sudoku solver as my very first hobby program. I was incredibly proud of that Sodoku program, and loved the motivation that doing my own project provided. It was a great way to dive in and learn. I'll be trying to reproduce that experience with machine learning.

But still, why machine learning? Just because it sounds like a fun way to get a computer to play a game? Well no, not really. A co-worker at Insitu recently left to go work for Google and will be working with the team that's helping to drive adoption of deep learning throughout Google. That certainly got me thinking about it more, but still, that's not the why either. It did lead me to read an excellent N.Y. Times article about Google's AI work, which led to yet more thinking. In the end, the full "why" is a confluence of two factors.

The first is climate change. It is extremely important to me that we do everything within our power to address the issue. I believe it's the "World War II" of our generation -- something for which the whole country should throw all of our efforts towards for as long as necessary, so that our children, and grandchildren, and great, great, great grandchildren will have the possibility for a bright and happy future. It's an issue that has weaved its way into many of my personal choices, such as becoming vegetarian (mostly), buying a Nissan Leaf, added insulating to our house, vacationing close to home, and even our choice not to have a second child. Climate change hasn't been the only consideration, or even necessarily the first, but its always been on the list.

I think personal choices matter, but am well aware that they can only accomplish so much. The nature of climate change requires that in the long run any solution must be achieved through global politics, with the whole of the planet agreeing to work together. But in the short run, there's still a great deal to be done technologically to help grease the skids -- to ensure that the path to a better future is as frictionless and easy to move towards as possible. These are things that groups of people working towards a common goal (i.e. a company) can accomplish. So I've been starting to search for a new field in which to apply my skills, and for ways to amplify the (positive) climate impact of my future work.

The second factor in "Why machine learning?" is that currently, I'm just a programmer and a team leader/manager for a small team working on a relatively basic product. I think I've become a fairly good programmer, but I am at best a big fish swimming in a small pond. I am not amazing or outstanding (or egotistical) enough to think that I will ever be at the true cutting edge of technology, solving mindbogglingly difficult problems for the very first time. I've known enough genuinely brilliant people to know that I can not count myself among their number. Still, even knowing that I may not have what it takes to be a leader in a field, I know I'm smart enough, and capable enough to be a fast follower. I'll give myself that much credit, at least.

I think machine learning, particularly deep learning has advanced to the point where it needs a healthy crop of "fast follower" types. It's very powerful, and has really advanced in the last few years due to algorithmic improvements, but also due to improvements in GPU computing, and in large-scale cloud computing. The learning phase of machine learning is quite data and compute intensive, yet once trained, the resource requirements are much, much lower. That, shall we say, "elasticity" in resource requirements can really benefit from the cloud cost model. Cloud computing has granted machine learning the sort of scale where it can accomplish things that traditional programming has extreme difficulty with, particularly speech recognition, translation, and image recognition.

I'm fairly convinced that ML is poised to revolutionize a number of fields over the next decade or so. I think it's a huge, powerful hammer of a technique, yet relatively few people currently know how to wield it. Yet the tools are there. The ground's been laid. It just hasn't occurred to most programmers that they could pick it up.

I've not dived in enough to know with certainty, but first impressions (i.e. a few days worth of reading) suggests that getting machine learning to work reasonably well is perhaps just a few times harder than getting a RDMS to work well. You have to learn a new paradigm, and there's lots of important details to learn before you can achieve anything ambitious, but you can put the tools to work without needing to learn anything too mind-bendingly difficult. Where the comparison breaks down is that ML will often need large amounts of carefully labeled training data, which is quite a lot harder to acquire than just a bunch of hard drives and a high-end server. Still, if you can overcome that requirement, I think that ML may be the sort of technique that eventually most programmers will have in their toolbelt, rather that always requiring a Ph.D'd specialist. That "eventually" may even be now, as far as potential goes.

So my goal is to learn enough about machine learning that I can confidently apply it when suitable, ideally without needing to become such a specialist that is the only tool I know how to wield. At the least, I want to learn enough that I can explain in detail why or why not various ML techniques can be applied to a given situation, and then bring in a specialist to help as appropriate.

Then, with the shiny new hammer in my belt, I want to go looking for domains where there really are a bunch of nails just waiting to be struck -- particularly domains which could potentially have a high impact on climate change. From where I stand right now, I think there's likely to be a lot of them.