Predictive Analytics For Dummies®

Published by: John Wiley & Sons, Inc., 111 River Street., Hoboken, NJ 07030-5774, www.wiley.com

Copyright © 2014 by John Wiley & Sons, Inc., Hoboken, New Jersey

Media and software compilation copyright © 2014 by John Wiley & Sons, Inc. All rights reserved.

Published simultaneously in Canada

No part of this publication may be reproduced, stored in a retrieval system or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning or otherwise, except as permitted under Sections 107 or 108 of the 1976 United States Copyright Act, without the prior written permission of the Publisher. Requests to the Publisher for permission should be addressed to the Permissions Department, John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030, (201) 748-6011, fax (201) 748-6008, or online at http://www.wiley.com/go/permissions.

Trademarks: Wiley, For Dummies, the Dummies Man logo, Dummies.com, Making Everything Easier, and related trade dress are trademarks or registered trademarks of John Wiley & Sons, Inc. and may not be used without written permission. All other trademarks are the property of their respective owners. John Wiley & Sons, Inc. is not associated with any product or vendor mentioned in this book.

LIMIT OF LIABILITY/DISCLAIMER OF WARRANTY: THE PUBLISHER AND THE AUTHOR MAKE NO REPRESENTATIONS OR WARRANTIES WITH RESPECT TO THE ACCURACY OR COMPLETENESS OF THE CONTENTS OF THIS WORK AND SPECIFICALLY DISCLAIM ALL WARRANTIES, INCLUDING WITHOUT LIMITATION WARRANTIES OF FITNESS FOR A PARTICULAR PURPOSE. NO WARRANTY MAY BE CREATED OR EXTENDED BY SALES OR PROMOTIONAL MATERIALS. THE ADVICE AND STRATEGIES CONTAINED HEREIN MAY NOT BE SUITABLE FOR EVERY SITUATION. THIS WORK IS SOLD WITH THE UNDERSTANDING THAT THE PUBLISHER IS NOT ENGAGED IN RENDERING LEGAL, ACCOUNTING, OR OTHER PROFESSIONAL SERVICES. IF PROFESSIONAL ASSISTANCE IS REQUIRED, THE SERVICES OF A COMPETENT PROFESSIONAL PERSON SHOULD BE SOUGHT. NEITHER THE PUBLISHER NOR THE AUTHOR SHALL BE LIABLE FOR DAMAGES ARISING HEREFROM. THE FACT THAT AN ORGANIZATION OR WEBSITE IS REFERRED TO IN THIS WORK AS A CITATION AND/OR A POTENTIAL SOURCE OF FURTHER INFORMATION DOES NOT MEAN THAT THE AUTHOR OR THE PUBLISHER ENDORSES THE INFORMATION THE ORGANIZATION OR WEBSITE MAY PROVIDE OR RECOMMENDATIONS IT MAY MAKE. FURTHER, READERS SHOULD BE AWARE THAT INTERNET WEBSITES LISTED IN THIS WORK MAY HAVE CHANGED OR DISAPPEARED BETWEEN WHEN THIS WORK WAS WRITTEN AND WHEN IT IS READ.

For general information on our other products and services, please contact our Customer Care Department within the U.S. at 877-762-2974, outside the U.S. at 317-572-3993, or fax 317-572-4002. For technical support, please visit www.wiley.com/techsupport.

Wiley publishes in a variety of print and electronic formats and by print-on-demand. Some material included with standard print versions of this book may not be included in e-books or in print-on-demand. If this book refers to media such as a CD or DVD that is not included in the version you purchased, you may download this material at http://booksupport.wiley.com. For more information about Wiley products, visit www.wiley.com.

Library of Congress Control Number: 2013954216

ISBN 978-1-118-72896--3 (pbk); ISBN 978-1-118-72941-0 (ebk); ISBN 978-1-118-72920-5 (ebk)

Manufactured in the United States of America

10 9 8 7 6 5 4 3 2 1

Predictive Analytics For Dummies

Visit www.dummies.com/cheatsheet/predictiveanalytics to view this book's cheat sheet.

Table of Contents

Introduction

About This Book

Foolish Assumptions

Icons Used in This Book

Beyond the Book

Where to Go from Here

Part I: Getting Started with Predictive Analytics

Chapter 1: Entering the Arena

Exploring Predictive Analytics

Mining data

Highlighting the model

Adding Business Value

Endless opportunities

Empowering your organization

Starting a Predictive Analytic Project

Business knowledge

Data-science team and technology

The Data

Surveying the Marketplace

Responding to big data

Working with big data

Chapter 2: Predictive Analytics in the Wild

Online Marketing and Retail

Recommender systems

Implementing a Recommender System

Collaborative filtering

Content-based filtering

Hybrid recommender systems

Target Marketing

Targeting using predictive modeling

Uplift modeling

Predictive Analytics Fight Fraud and Crime

Content and Text Analytics

Chapter 3: Exploring Your Data Types and Associated Techniques

Recognizing Your Data Types

Structured and unstructured data

Static and streamed data

Identifying Data Categories

Attitudinal data

Behavioral data

Demographic data

Generating Predictive Analytics

Data-driven analytics

User-driven analytics

Connecting to Related Disciplines

Statistics

Data mining

Machine learning

Chapter 4: Complexities of Data

Finding Value in Your Data

Delving into your data

Data validity

Data variety

Constantly Changing Data

Data velocity

High volume of data

Complexities in Searching Your Data

Keyword-based search

Semantic-based search

Differentiating Business Intelligence from Big-Data Analytics

Visualization of Raw Data

Identifying data attributes

Exploring data visualization

Tabular visualizations

Bar charts

Pie charts

Graph charts

Word clouds as representations

Line graphs

Flocking birds representation

Part II: Incorporating Algorithms in Your Models

Chapter 5: Applying Models

Modeling Data

Models and simulation

Categorizing models

Describing and summarizing data

Making better business decisions

Healthcare Analytics Case Studies

Google search queries as epidemic predictors

Cancer survivability predictors

Social and Marketing Analytics Case Studies

Tweets as predictors for the stock market

Target store predicts pregnant women

Twitter-based predictors of earthquakes

Twitter-based predictors of political campaign outcomes

Chapter 6: Identifying Similarities in Data

Explaining Data Clustering

Motivation

Converting Raw Data into a Matrix

Creating a matrix of terms in documents

Term selection

Identifying K-Groups in Your Data

K-means clustering algorithm

Clustering by nearest neighbors

Finding Associations Among Data Items

Applying Biologically Inspired Clustering Techniques

Birds flocking

Ant colonies

Chapter 7: Predicting the Future Using Data Classification

Explaining Data Classification

Lending

Marketing

Healthcare

What’s next?

Introducing Data Classification to Your Business

Exploring the Data-Classification Process

Using Data Classification to Predict the Future

Decision trees

Support vector machine

Naïve Bayes classification algorithm

Neural networks

The Markov Model

Linear regression

Ensemble Methods to Boost Prediction Accuracy

Part III: Developing a Roadmap

Chapter 8: Convincing Your Management to Adopt Predictive Analytics

Making the Business Case

Benefits to the business

Gathering Support from Stakeholders

Working with your sponsors

Getting business and operations buy-in

Getting IT buy-in

Rapid prototyping

Presenting Your Proposal

Chapter 9: Preparing Data

Listing the Business Objectives

Identifying related objectives

Collecting user requirements

Processing Your Data

Identifying the data

Cleaning the data

Generating any derived data

Reducing the dimensionality of your data

Structuring Your Data

Extracting, transforming and loading your data

Keeping the data up to date

Outlining testing and test data

Chapter 10: Building a Predictive Model

Getting Started

Defining your business objectives

Preparing your data

Choosing an algorithm

Developing and Testing the Model

Developing the model

Testing the model

Evaluating the model

Going Live with the Model

Deploying the model

Monitoring and maintaining the model

Chapter 11: Visualization of Analytical Results

Visualization As a Predictive Tool

Why visualization matters

Getting the benefits of visualization

Dealing with complexities

Evaluating Your Visualization

How relevant is this picture?

How interpretable is the picture?

Is the picture simple enough?

Does the picture lead to new insights?

Visualizing Your Model’s Analytical Results

Visualizing hidden groupings in your data

Visualizing data classification results

Visualizing outliers in your data

Visualization of Decision Trees

Visualizing predictions

Other Types of Visualizations in Predictive Analytics

Bird-flocking behavior data visualization

Part IV: Programming Predictive Analytics

Chapter 12: Creating Basic Prediction Examples

Installing the Software Packages

Installing Python

Installing the machine-learning module

Installing the dependencies

Preparing the Data

Getting the sample dataset

Labeling your data

Making Predictions Using Classification Algorithms

Creating a supervised learning model with SVM

Creating a supervised learning model with logistic regression

Comparing two classification models

Chapter 13: Creating Basic Examples of Unsupervised Predictions

Getting the Sample Dataset

Using Clustering Algorithms to Make Predictions

Comparing two clustering models

Creating an unsupervised learning model with K-means

Creating an unsupervised learning model with DBSCAN

Chapter 14: Predictive Modeling with R

Programming in R

Installing R

Installing RStudio

Getting familiar with the environment

Learning just a bit of R

Making Predictions Using R

Predicting using regression

Using classification to predict

Chapter 15: Avoiding Analysis Traps

Data Challenges

Outlining the limitations of the data

Dealing with extreme cases (outliers)

Data smoothing

Curve fitting

Keeping the assumptions to a minimum

Analysis Challenges

Supervised analytics

Relying on only one analysis

Describing the limitations of the model

Avoiding non-scalable models

Scoring your predictions accurately

Chapter 16: Targeting Big Data

Major Technological Trends in Predictive Analytics

Exploring predictive analytics as a service

Aggregating distributed data for analysis

Real-time data-driven analytics

Applying Open-Source Tools to Big Data

Apache Hadoop

Apache Mahout

Building a Rapid Prototype of Your Predictive Analytics Model

Prototyping for predictive analytics

Testing your predictive analytics model

Part V: The Part of Tens

Chapter 17: Ten Reasons to Implement Predictive Analytics

Outlining Business Goals

Knowing Your Data

Organizing Your Data

Satisfying Your Customers

Reducing Operational Costs

Increasing Returns on Investments (ROI)

Increasing Confidence

Making Informed Decisions

Gaining Competitive Edge

Improving the Business

Chapter 18: Ten Steps to Build a Predictive Analytic Model

Building a Predictive Analytics Team

Getting business expertise on board

Firing up IT and math expertise

Setting the Business Objectives

Preparing Your Data

Sampling Your Data

Avoiding “Garbage In, Garbage Out”

Keeping it simple isn’t stupid

Data preparation puts the good stuff in

Creating Quick Victories

Fostering Change in Your Organization

Building Deployable Models

Evaluating Your Model

Updating Your Model

About the Authors

Cheat Sheet

More Dummies Products

End User License Agreement

Guide

Table of Contents

Begin Reading

Pages

1

2

3

5

6

8

9

10

11

12

13

14

15

16

17

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

40

41

42

44

45

46

47

48

49

50

51

52

53

54

55

58

59

60

61

62

63

64

65

66

67

68

69

70

71

72

73

74

76

77

78

79

80

81

82

83

84

85

86

87

88

89

90

91

92

93

94

95

96

97

98

99

100

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

135

136

137

138

139

140

141

142

143

145

146

147

148

149

150

151

152

154

155

156

157

158

159

160

161

162

163

164

165

166

168

169

170

171

172

173

174

175

176

178

179

180

181

182

183

184

185

186

187

188

189

190

191

192

193

194

195

196

197

198

199

200

201

202

203

204

205

206

207

208

209

210

211

212

213

214

215

216

217

218

219

220

221

222

223

224

225

226

227

228

230

231

232

234

235

236

237

238

239

240

241

242

243

244

245

246

247

248

249

250

251

252

253

254

255

256

257

258

259

260

261

262

263

264

265

266

267

268

269

270

271

272

273

274

276

277

278

279

280

281

282

283

284

285

286

287

288

289

290

291

292

293

294

296

297

298

299

300

301

302

303

304

305

306

307

308

309

310

311

312

313

314

315

316

317

318

320

321

322

323

324

325

326

327

328

329

330

345

346

347

348

Introduction

Predictive Analytics is the art and science of using data to make better informed decisions. Predictive analytics helps you uncover hidden patterns and relationships in your data that can help you predict with greater confidence what may happen in the future, and provide you with valuable, actionable insights for your organization.

About This Book

Our goal was to make this complex subject as practical as possible, in a way that appeals to everyone from technological experts to non-technical level business strategists.

The subject is complex because it is not really just one subject. It is the combination of at least a few multifaceted fields: data mining, statistics and mathematics.

Data mining requires an understanding of machine learning and information retrieval. On top of this, mathematics and statistics must be applied to your business domain; be it marketing, actuary service, fraud, crime, or banking.

Most of the current materials on predictive analytics are pretty difficult to read if you don't already have a background in some of the aforementioned subjects. They are filled with complex mathematical equations and modeling techniques. Or, they are at a high level with specific use cases but with little guidance regarding implementation. We tried to include both, while trying to keep a wide spectrum of readers engaged.

The focus of this book will be developing a roadmap for implementing predictive analytics within your organization. Its intended audience is the larger community of business managers, business analysts, data scientists, and information technology professionals.

Maybe you are a business manager and you have heard the buzz about predictive analytics. Maybe you've been working with data mining and you want to add predictive analytics to your skill set. Maybe you know R or Python, but you're totally new to predictive analytics. If this sounds like you, then this book will be a good fit. Even if you have no experience analyzing data, but want or need to derive greater value from your organization’s data, you can also find something of value in this book.

Foolish Assumptions

Without oversimplifying, we have tried to explain technical concepts in non-technical terms, tackling each topic from the ground up.

Even if you are an experienced practitioner, you should find something new, and at the very least, you will gain validation for what you already know, and guidance for establishing best practices.

We also hope to have contributed a few concepts and ideas for the very first time in a major publication like this. For example we explain how you can apply biologically inspired algorithms to predictive analytics.

We assume that the reader will not be a programmer. The code presented in this book is very brief and easy to follow. Readers of all programming levels will benefit from this book, because it is more about learning the process of predictive analytics rather than learning a programming language.

Icons Used in This Book

The following icons in the margins indicate highlighted material that we thought could be of interest to you. Next, we describe the meaning of each icon that is used in this book.

The tips are ideas we would like you to take note of. This is usually practical advice you can apply for that given topic.

This icon is rarely used in this book. We may have used it only once or twice in the entire book. The intent is to save you time by bringing to your attention some common pitfalls that you are better off avoiding.

We have made sincere efforts to steer away from the technical stuff. But when we had no choice we made sure to let you know. So if you don’t care too much about the technical stuff you can easily skip this part and you won’t miss much. If the technical stuff is your thing, then you may find these sections fascinating.

This is something we would like you to take a special note of. This is a concept or idea we thought it was important for you know and remember. An example of this would be a best practice we think it is noteworthy.

Beyond the Book

A lot of extra content that is not in this book is available at www.dummies.com. Go online to find the following:

Online articles covering additional topics at

  www.dummies.com/extras/predictiveanalytics

Here you will find articles about advanced topics in predictive analytics, and an overview about learning data mining with R. We have also included top ten qualities for your data science team.

The Cheat Sheet for this book is at

  www.dummies.com/cheatsheet/predictiveanalytics

Here you’ll find the necessary steps needed to build a predictive analytics model and some cases studies of predictive analytics.

Updates to this book, if we have any, are also available at

  www.dummies.com/extras/predictiveanalytics

Find a bonus chapter covering major vendors in the predictive analytics industry on the downloads tab at

  www.dummies.com/go/predictiveanalytics

Where to Go from Here

Let’s start making some predictions! You can apply predictive analytics to virtually every business domain. Right now there is explosive growth in predictive analytics’ market, and this is just the beginning. The arena is wide open, and the possibilities are endless.

Part I

Getting Started with Predictive Analytics

Visit www.dummies.com for great Dummies content online.

In this part …

Exploring predictive analytics

Identifying uses

Classifying data

Presenting information

Visit www.dummies.com for great Dummies content online.

Chapter 1

Entering the Arena

In This Chapter

arrow Explaining the building blocks

arrow Probing capabilities

Lesen Sie weiter in der vollständigen Ausgabe!

Lesen Sie weiter in der vollständigen Ausgabe!

Lesen Sie weiter in der vollständigen Ausgabe!

Lesen Sie weiter in der vollständigen Ausgabe!

Lesen Sie weiter in der vollständigen Ausgabe!

Lesen Sie weiter in der vollständigen Ausgabe!

Lesen Sie weiter in der vollständigen Ausgabe!

Lesen Sie weiter in der vollständigen Ausgabe!

Lesen Sie weiter in der vollständigen Ausgabe!

Lesen Sie weiter in der vollständigen Ausgabe!

Lesen Sie weiter in der vollständigen Ausgabe!

Lesen Sie weiter in der vollständigen Ausgabe!

Lesen Sie weiter in der vollständigen Ausgabe!

Lesen Sie weiter in der vollständigen Ausgabe!

Lesen Sie weiter in der vollständigen Ausgabe!

Lesen Sie weiter in der vollständigen Ausgabe!

Lesen Sie weiter in der vollständigen Ausgabe!

Lesen Sie weiter in der vollständigen Ausgabe!

Lesen Sie weiter in der vollständigen Ausgabe!

Lesen Sie weiter in der vollständigen Ausgabe!

Lesen Sie weiter in der vollständigen Ausgabe!

Lesen Sie weiter in der vollständigen Ausgabe!

Lesen Sie weiter in der vollständigen Ausgabe!

Lesen Sie weiter in der vollständigen Ausgabe!

Lesen Sie weiter in der vollständigen Ausgabe!