Thursday, September 12, 2013

Kaggle Flourishes by Embracing the Competitive Nature of the Data Scientist

The emergence and success of Kaggle, a website dedicated to matching number crunchers with companies in need, shows us that no matter how challenging the problem there is always someone willing and eager to tackle it head on.

We’ve been hearing more and more about Kaggle in various capacities: candidates posting their Kaggle ranks on their LinkedIn profiles, students or recent grads using the site to gain experience with unstructured data, or even companies asking for candidates’ Kaggle rankings in job postings. For an aspiring data scientist with little to no experience this could be a invaluable tool to gain experience with real-world big, messy data.

Kaggle has grown in popularity from 25,000 to over 100,000 members in a little over a year and received attention from numerous publications such as The New York Times and Forbes about its unique service. Quantitative professionals from all over the globe can compete for prize money by offering intellectual property including any algorithms, models, solutions etc.

In addition to earning money for their services, Quants have the opportunity to hone their skills and see how their solutions and skills rank against other members. For companies who cannot afford or do not need a full time analytics staff, Kaggle will match you with a team based on your specific initiative that you can hire on a contract basis.  According to the founders of Kaggle, their intention was to inspire great performance by inciting competition amongst the members.

The competitions are divided into sections, with one dedicated to “Getting Started” which offers entries and students looking to test their skills on real data sets an opportunity to see how they compare to working data scientists worldwide. There have also been competitions from Belkin, Amazon and Expedia for more advanced participants.

Most recently, Facebook is using Kaggle to host a recruiting competition. Could this site completely revolutionize the way Quants learn, share, compete or job-hunt within the community? As anyone keeping abreast of Big Data news will tell you, we’re moving into the age of digital everything- socializing, interviewing, learning- why not digital sports for the data scientists too?