2011 Data Miner Survey – Relatório sobre práticas em Mineração de Dados

No ano de 2011 foi realizada uma pesquisa pela Rexer Analytics sobre as principais práticas de mineração de dados, bem como as tendências. De forma geral pelos highlights dá para se perceber que apesar da evolução das técnicas muito do que está em ‘produção’ hoje tem a ver com o básico: Árvore de Decisão, Análise de Cluster e Regressão.

Isso mostra que deve haver um trabalho de base forte para consolidação da mineração de dados no cenário nacional; em especial, na área acadêmica na qual há a apresentação de conceitos avançados sendo que na prática há pouco sendo feito.

 Alguns dos pontos principais elencados pela pesquisa:

FIELDS & GOALS:  Data miners work in a diverse set of fields.  CRM / Marketing has been the #1 field in each of the past five years.  Fittingly, “improving the understanding of customers”, “retaining customers” and other CRM goals continue to be the goals identified by the most data miners.

ALGORITHMS:  Decision trees, regression, and cluster analysis continue to form a triad of core algorithms for most data miners.  However, a wide variety of algorithms are being used.   A third of data miners currently use text mining and another third plan to in the future.  Text mining is most often used to analyze customer surveys and blogs/social media.

TOOLS:  R continued its rise this year and is now being used by close to half of all data miners (47%).  R users report preferring it for being free, open source, and having a wide variety of algorithms.  Many people also cited R’s flexibility and the strength of the user community.  In the 2011 survey we asked R users to tell us more about their use of R.  Read the R user comments about why these use R (pros), the cons of using R, why they select their R interface, and how they use R in conjuction with other tools.  STATISTICA is selected as the primary data mining tool by the most data miners (17%).  Data miners report using an average of 4 software tools overall.  STATISTICA, KNIME, Rapid Miner and Salford Systems received the strongest satisfaction ratings in 2011.

TECHNOLOGY:  Data Mining most often occurs on a desktop or laptop computer, and  requently the data is stored locally.  Model scoring typically happens using the same software used to develop models.

VISUALIZATION:  Data miners frequently use data visualization techniques.  More than four in five use them to explain results to others.  MS Office is the most often used tool  for data visualization.  Extensive use of data visualization is less prevalent in the Asia-Pacific region than other parts of the world.

ANALYTIC CAPABILITY & SUCCESS:  Only 12% of corporate respondents rate their company as having very high analytic sophistication.  However, companies with better analytic capabilities are outperforming their peers.  Respondents report analyzing analytic success via Return on Investment (ROI), and analyzing the predictive validity or accuracy of their models.  Challenges to measuring analytic success include client or user cooperation and data availability / quality.

FUTURE:  Data miners are optimistic about continued growth in data mining adoption and the positive impact data mining will have.  As in previous years, data miners see growth in the number of projects they will be conducting.  And growth in data mining adoption is the number one “future trend” identified.  Participants pointed out that care must be taken to protect privacy when conducting data mining.  Data miners also shared many examples of the positive impact they feel data mining can have to benefit society.  Health / medical advances was the area of positive impact identified by the most data miners.