Beginning with Machine Learning and AI

Posted on by

Even though Machine Learning and AI have been around for decades it feels like day one. This is especially true for topics related to Artificial Neural Networks, Deep Neural Networks a.k.a. Deep Learning, etc.. The excitement and enthusiasm in the community of machine learning practitioners and “learners” is palpable and infectious! It’s hard not to get drawn in once you start looking at it.

Be a beginner in Machine Learning at any level

It’s possible to be a beginner in machine learning no matter what your background is (or even how long you have been working with it). It’s one of the most accessible “scientific subjects” for a computer enthusiast. Is it easy? No, it’s not easy! But, It’s possible for nearly anyone to learn how to “do” something with it, including amateur hobbyist to PhD’s. It’s wide open to new ideas and applications.

I’m a beginner! I have a science PhD and a strong mathematics background but really I’m new to most of machine learning. It’s great to be a beginner. You can have crazy ideas and really have fun trying them out. As a beginner your not held back by preconceived ideas and restrictions. Many of the Machine Learning algorithms and methods are easy enough to implement that you can try new ideas and just see what happens. I’m hooked!

I love this quote from Geoffrey Hinton (one of the most persistent and respected researchers in the field )

“If you think it’s a really good idea and other people tell you it is complete nonsense then you know you are really on to something”

He was referring to more advanced work but really that just applies in general. Don’t let your curiosity and creativity be held back by what others think! Keep the mindset of a “beginner”.

Begin

A very curious thing about the subject of machine learning is that the barrier to entry is really not that high. There is a high school student winning machine learning competitions on Kaggle! There is also massive cross-over from fields outside of the disciplines that founded machine learning (statistics, mathematics and computer science). The range of people “practicing” and contributing to machine learning is from students and hobbyists to PhD’s in a wide varieties of fields.

Machine Learning is being accepted in nearly every field of work that make use of computers in any way. Virtually all fields of science, business, finance, economics, medicine, manufacturing, entertainment, advertising, government, politics … it seems everyone has something they are doing or would like to do. Some ideas and endeavors are more noble than others 🙂 but the curiosity and excitement for the work are the same for everyone.

I have been going to “meetup’s”, talks and conferences and there is always an air of discovery and curiosity around machine learning. These feelings are shared by individuals across a full range of experience, education, and skill levels. There is also a strong sense of community, sharing, and acceptance.

The landscape for machine learning seems like a large mountain range with many peaks and valleys. There are peaks that are hidden by clouds waiting to be explored and there are gentle sloping paths with hidden turns and surprises. The top researches are scaling those high hidden peaks but there are many paths that are becoming clear enough for even the curious hobbyist to travel and explore. With a little Python programming experience and something like Keras on top of TensorFlow even complex neural networks are reasonably easy to setup and run. (You wont do that on your first day 🙂 … just begin. The first thing you have to do is get close enought to that mountain range to see the paths.

Learning Machine Learning

Higher education for machine learning and AI is being provided not just by university PhD programs. Large and even small companies are hiring the brightest minds to act as researchers AND mentors. They and their students may have proven themselves by non-traditional means outside of formal college degrees.

A portfolio of a few good projects on GitHub or notable participation in a good open source project is becoming more valuable than a university degree to hiring managers looking for Machine Learning talent.

For someone new to machine learning that has a strong mathematics background and some programming experience, possibly from work in another field, things will be relatively easy. The field is well established but most of the core ideas are not too difficult. And, most importantly, there is an enormous amount of work to be done! Coming in with a fresh perspective can be an asset for doing serious research. There is a very large amount of cross-over from other fields. From what I’ve personally seen it looks to be rare to find people that are working in machine learning to have come in with a computer science background!

Machine learning is a whole new way to look at making computers do something useful. The idea is to train the computer to do something rather than explicitly program it. That is somewhat at odds with the traditional computer science curriculum!

For the “new” beginner

For the beginner wanting to learn machine learning, education resources are readily available. The keys are motivation and persistence. Have a reason to pursue it, then take baby steps and have fun with it. Your motivation can be simple curiosity and the desire to learn something new or more serious like maybe considering it as a career path. Maybe you have an idea for an app that would make you a millionaire! OK, how are you going to do that? You just start working on it, move forward! You may find that the journey opens up ides and opportunities you never considered. Don’t expect over-night success. Think “what do I want to be doing 3-5 years from now?”.

Beginners education has many approaches

The tradition education route with a college degree would be good but it’s not entirely necessary. There are some academic subjects that would be very helpful. The main thing is to acquire basic skills and learn the language of the field so you can read and understand what people working in machine learning are talking about.

Good courses would be,

  • Basic programming (Python!)
  • Calculus (needed mostly to understand “optimization”)
  • Probability and Statistics (One definition of Machine Learning is that “it is Statistics”)
  • Linear Algebra (Numerical Linear Algebra is the core computational )
    tool for most scientific computing including Machine Learning)

There are other subjects that could be nice to know but those are the core. Be warned that those courses can be taught from a theory perspective or an application perspective. Either is good, but unless you want to really “be a mathematician” then courses that say “Applied” at the beginning might be the better choice. Beyond those core subjects anything you like would be good to add to the mix to get your degree.

Online courses, books, and papers (do-it-yourself a-la-cart education)

Pick what you want to learn and dive in! If there had been resources like Coursera, edX, Udemy, Udacity, Safari Books OnLine, arXiv.org etc. available when I was young I probably never would have gone to school. [ I actually didn’t go anyway. I quite high school and was a starving musician for a while before I joined the Army to get money to go to college. I eventually got a degree in Mathematics and a PhD in Theoretical Chemistry. Never stop learning! ]

There are so many good resources on-line it’s hard to decide what to look at. Here’s a pretty good analysis of some of the offerings for Machine Learning. “Every Single Machine Learning Course on the Internet Ranked by your Reviews”. He also, has links for posts on similar rankings for coding classes, probability and statistics, data science and data visualization. I don’t necessarily agree with everything there but it’s a fascinating analysis. There is a lot of good beginner material listed!

I am a long time subscriber to O’Reilly Safari Books OnLine It’s expensive but I really enjoy it and feel I get my moneys worth every year. There are 10’s of thousands of books and videos on there and lots of them are very good (over a wide range of interests but mostly technical). It is interesting just to look at new book listings to see how many of them are about machine learning, Python etc.. You can get a feel for what is trending just from that. I can tell you Python and TensorFlow are pretty popular right now.

For really advanced material Cornell University arXiv.org is one of the best things ever to happen for research papers. People but pre-prints of papers there so that you can access them even if you don’t have access to a world class research library to get to journals where they eventually end up. [rant-on] It is appalling that commercial scientific journal publishers have so many important papers locked away behind pay-walls keeping them out of the hands of the public even though most of the research was paid for with public (government) funds [rand-off] There are over a million papers on arXiv.org! Fortunately Andrej Karpathy (currently director of AI at Tesla) has created a site called Arxiv Sanity Preserver. That is a community site for machine learning researches where papers are discussed and recommended. Current, and important research papers are available and discussed there!

I want to mention a great book that is available to read on-line that is a little advanced but looks really good. It has a section on the mathematics you need to understand Deep Learning. If you are just starting out that is probably too much but it is a good reality check for anyone that wants to pursue research or have a deep understanding of machine learning algorithms and methods.

Goodfellow-et-al-2016,
Deep Learning,
authors: Ian Goodfellow and Yoshua Bengio and Aaron Courville,
MIT Press,
http://www.deeplearningbook.org

First steps

That’s a lot for first steps! Start with learning a little Python you don’t really need to be a “great” Python programmer but that is definitely the best way to get started. You will be able to work interactively in Jupyter notebooks and load in machine learning tools and frameworks to do experiments.

Happy computing –dbk

Tags: , , , ,