The world's leading music streaming service, Spotify, has selected Hortonworks Data Platform (HDP) to further improve the personalised music experience it offers to its users.

HDP will be used to underpin Spotify's Hadoop infrastructure, said to be Europe's largest commercially-used cluster, with 690 nodes, while storing data from the company's 24 million active users and six million subscribers.

Spotify uses Hadoop to analyse its users' data and offer music recommendations that are based on what they have listened to in the past. It is now also offering bands and musicians commercial services that include advice on where to hold concerts, using its geo-location data to determine where an artist has a strong fan base.

Spotify launched in 2008 and began using Hadoop with a relatively small cluster of 30 nodes. However, since its launch there have been a number of other music streaming services entering the market and Spotify has made moves to scale up its use of Hadoop in order to differentiate itself from the competition by offering customers a more data-driven, personalised music experience.

Hortonworks will provide an integrated and tested Hadoop distribution and support Spotify's technical in-house team. It will also run semi-annual health assessments on Spotify's cluster.

"The cultural fit was an important factor in our selection and we have appreciated Hortonworks' relaxed, helpful and open approach," said Wouter de Bie, team lead for data infrastructure at Spotify.

"We were looking for a true partner relationship and the team at Hortonworks are committed to enabling the overall ecosystem - including the vendors we rely on - to leverage Hadoop. Their true open source approach and the work they have done to improve the Apache Hive data warehouse system also aligns well with our needs, as we use Hive extensively for ad-hoc queries and for the analysis of large data sets."

Computerworld UK met with Hortonworks' president Herb Cunitz last week, who explained that the deal with Spotify marks an important expansion into Europe for the US-based company. Hortonworks' European business accounts for approximately 10 percent of revenues at present - it hopes to increase this to 20 percent over the next 12 months.

"Spotify is undertaking some really innovative work in the data analytics field and realised the need for a deep level of open source Apache Hadoop domain experience and expertise.

"Hortonworks prides itself on the Hadoop pedigree of our many contributors and we very much look forward to applying this knowledge to help Spotify continue to break new ground and drive its business forward in creating a next generation data architecture," said Herb Cunitz, president, Hortonworks.