You’ll learn how to combine social web data, analysis techniques, and visualization to find what you’ve been looking for in the social haystack—as well as useful information you didn’t know existed.Each standalone chapter introduces techniques for mining data in different areas of the social Web, including blogs and email. All you need to get started is a programming background and a willingness to learn basic Python tools.Get a straightforward synopsis of the social web landscapeUse adaptable scripts on GitHub to harvest data from social network APIs such as Twitter, Facebook, LinkedIn, and Google+Learn how to employ easy-to-use Python tools to slice and dice the data you collectExplore social connections in microformats with the XHTML Friends NetworkApply advanced mining techniques such as TF-IDF, cosine similarity, collocation analysis, document summarization, and clique detectionBuild interactive visualizations with web technologies based upon HTML5 and JavaScript toolkits"A rich, compact, useful, practical introduction to a galaxy of tools, techniques, and theories for exploring structured and unstructured data." --Alex Martelli, Senior Staff Engineer, Google

    As the recent collapse on Wall Street shows, we are often ill-equipped to deal with uncertainty and risk. Yet every day we base our personal and business plans on uncertainties, whether they be next month's sales, next year's costs, or tomorrow's stock price. In The Flaw of Averages, Sam Savage-known for his creative exposition of difficult subjects- describes common avoidable mistakes in assessing risk in the face of uncertainty. Along the way, he shows why plans based on average assumptions are wrong, on average, in areas as diverse as healthcare, accounting, the War on Terror, and climate change. In his chapter on Sex and the Central Limit Theorem, he bravely grasps the literary third rail of gender differences.Instead of statistical jargon, Savage presents complex concepts in plain English. In addition, a tightly integrated web site contains numerous animations and simulations to further connect the seat of the reader's intellect to the seat of their pants.The Flaw of Averages typically results when someone plugs a single number into a spreadsheet to represent an uncertain future quantity. Savage finishes the book with a discussion of the emerging field of Probability Management, which cures this problem though a new technology that can pack thousands of numbers into a single spreadsheet cell.Praise for The Flaw of Averages"Statistical uncertainties are pervasive in decisions we make every day in business, government, and our personal lives. Sam Savage's lively and engaging book gives any interested reader the insight and the tools to deal effectively with those uncertainties. I highly recommend The Flaw of Averages." --William J. Perry, Former U.S. Secretary of Defense"Enterprise analysis under uncertainty has long been an academic ideal. . . . In this profound and entertaining book, Professor Savage shows how to make all this practical, practicable, and comprehensible." ---Harry Markowitz, Nobel Laureate in Economics

    Employing an irresistible cast of dragon-riding Vikings, lizard-throwing giants, and feuding aliens, the renowned illustrator Grady Klein and the award-winning statistician Alan Dabney teach you how to collect reliable data, make confident statements based on limited information, and judge the usefulness of polls and the other numbers that you're bombarded with every day. If you want to go beyond the basics, they've created the ultimate resource: "The Math Cave," where they reveal the more advanced formulas and concepts.Timely, authoritative, and hilarious, The Cartoon Introduction to Statistics is an essential guide for anyone who wants to better navigate our data-driven world.

    Employing a methodology that is primarily learning by example and “thinking with our eyes,” this manual features graphs and practical analytical techniques that can be applied to a broad range of data analysis tools—including the most commonly used Microsoft Excel. This approach is particularly valuable to those who need to make sense of quantitative business data by discerning meaningful patterns, trends, relationships, and exceptions that reveal business performance, potential problems and opportunities, and hints about the future. It provides practical skills that are useful to managers at all levels and to those interested in keeping a keen eye on their business.

    It explains what makes some graphs succeed while others fail, how to make high-quality figures from data using powerful and reproducible methods, and how to think about data visualization in an honest and effective way.Data Visualization builds the reader's expertise in ggplot2, a versatile visualization library for the R programming language. Through a series of worked examples, this accessible primer then demonstrates how to create plots piece by piece, beginning with summaries of single variables and moving on to more complex graphics. Topics include plotting continuous and categorical variables; layering information on graphics; producing effective "small multiple" plots; grouping, summarizing, and transforming data for plotting; creating maps; working with the output of statistical models; and refining plots to make them more comprehensible.Effective graphics are essential to communicating ideas and a great way to better understand data. This book provides the practical skills students and practitioners need to visualize quantitative data and get the most out of their research findings.Provides hands-on instruction using R and ggplot2Shows how the "tidyverse" of data analysis tools makes working with R easier and more consistentIncludes a library of data sets, code, and functions

    In Standard Deviations, economics professor Gary Smith walks us through the various tricks and traps that people use to back up their own crackpot theories. Sometimes, the unscrupulous deliberately try to mislead us. Other times, the well-intentioned are blissfully unaware of the mischief they are committing. Today, data is so plentiful that researchers spend precious little time distinguishing between good, meaningful indicators and total rubbish. Not only do others use data to fool us, we fool ourselves.With the breakout success of Nate Silver’s The Signal and the Noise, the once humdrum subject of statistics has never been hotter. Drawing on breakthrough research in behavioral economics by luminaries like Daniel Kahneman and Dan Ariely and taking to task some of the conclusions of Freakonomics author Steven D. Levitt, Standard Deviations demystifies the science behind statistics and makes it easy to spot the fraud all around.

    R has traditionally been difficult for non-statisticians to learn, and most R books assume far too much knowledge to be of help. R for Everyone is the solution. Drawing on his unsurpassed experience teaching new users, professional data scientist Jared P. Lander has written the perfect tutorial for anyone new to statistical programming and modeling. Organized to make learning easy and intuitive, this guide focuses on the 20 percent of R functionality you'll need to accomplish 80 percent of modern data tasks. Lander's self-contained chapters start with the absolute basics, offering extensive hands-on practice and sample code. You'll download and install R; navigate and use the R environment; master basic program control, data import, and manipulation; and walk through several essential tests. Then, building on this foundation, you'll construct several complete models, both linear and nonlinear, and use some data mining techniques. By the time you're done, you won't just know how to write R programs, you'll be ready to tackle the statistical problems you care about most. COVERAGE INCLUDES - Exploring R, RStudio, and R packages - Using R for math: variable types, vectors, calling functions, and more - Exploiting data structures, including data.frames, matrices, and lists - Creating attractive, intuitive statistical graphics - Writing user-defined functions - Controlling program flow with if, ifelse, and complex checks - Improving program efficiency with group manipulations - Combining and reshaping multiple datasets - Manipulating strings using R's facilities and regular expressions - Creating normal, binomial, and Poisson probability distributions - Programming basic statistics: mean, standard deviation, and t-tests - Building linear, generalized linear, and nonlinear models - Assessing the quality of models and variable selection - Preventing overfitting, using the Elastic Net and Bayesian methods - Analyzing univariate and multivariate time series data - Grouping data via K-means and hierarchical clustering - Preparing reports, slideshows, and web pages with knitr - Building reusable R packages with devtools and Rcpp - Getting involved with the R global community

    It is a ‘ must’ book for those in every branch of science . . . in addition, economists, politicians, statesmen, and businessmen cannot afford to overlook cybernetics and its tremendous, even terrifying implications. "It is a beautifully written book, lucid, direct, and despite its complexity, as readable by the layman as the trained scientist." -- John B. Thurston, "The Saturday Review of Literature" Acclaimed one of the "seminal books . . . comparable in ultimate importance to . . . Galileo or Malthus or Rousseau or Mill," "Cybernetics" was judged by twenty-seven historians, economists, educators, and philosophers to be one of those books published during the "past four decades", which may have a substantial impact on public thought and action in the years ahead." -- Saturday Review

    But with the increasing power of visualization software today, scientists, engineers, and business analysts often have to navigate a bewildering array of visualization choices and options.This practical book takes you through many commonly encountered visualization problems, and it provides guidelines on how to turn large datasets into clear and compelling figures. What visualization type is best for the story you want to tell? How do you make informative figures that are visually pleasing? Author Claus O. Wilke teaches you the elements most critical to successful data visualization.Explore the basic concepts of color as a tool to highlight, distinguish, or represent a valueUnderstand the importance of redundant coding to ensure you provide key information in multiple waysUse the book's visualizations directory, a graphical guide to commonly used types of data visualizationsGet extensive examples of good and bad figuresLearn how to use figures in a document or report and how employ them effectively to tell a compelling story

    The story of information begins in a time profoundly unlike our own, when every thought and utterance vanishes as soon as it is born. From the invention of scripts and alphabets to the long-misunderstood talking drums of Africa, Gleick tells the story of information technologies that changed the very nature of human consciousness. He provides portraits of the key figures contributing to the inexorable development of our modern understanding of information: Charles Babbage, the idiosyncratic inventor of the first great mechanical computer; Ada Byron, the brilliant and doomed daughter of the poet, who became the first true programmer; pivotal figures like Samuel Morse and Alan Turing; and Claude Shannon, the creator of information theory itself. And then the information age arrives. Citizens of this world become experts willy-nilly: aficionados of bits and bytes. And we sometimes feel we are drowning, swept by a deluge of signs and signals, news and images, blogs and tweets. The Information is the story of how we got here and where we are heading.

    Several resources exist for individual pieces of this data science stack, but only with the Python Data Science Handbook do you get them all—IPython, NumPy, Pandas, Matplotlib, Scikit-Learn, and other related tools.Working scientists and data crunchers familiar with reading and writing Python code will find this comprehensive desk reference ideal for tackling day-to-day issues: manipulating, transforming, and cleaning data; visualizing different types of data; and using data to build statistical or machine learning models. Quite simply, this is the must-have reference for scientific computing in Python.With this handbook, you’ll learn how to use: * IPython and Jupyter: provide computational environments for data scientists using Python * NumPy: includes the ndarray for efficient storage and manipulation of dense data arrays in Python * Pandas: features the DataFrame for efficient storage and manipulation of labeled/columnar data in Python * Matplotlib: includes capabilities for a flexible range of data visualizations in Python * Scikit-Learn: for efficient and clean Python implementations of the most important and established machine learning algorithms

    Now with the second edition, readers will find information on key new topics such as neural networks and statistical pattern recognition, the theory of machine learning, and the theory of invariances. Also included are worked examples, comparisons between different methods, extensive graphics, expanded exercises and computer project topics.An Instructor's Manual presenting detailed solutions to all the problems in the book is available from the Wiley editorial department.

    Jill Lepore, best-selling author of These Truths, came across the company’s papers in MIT’s archives and set out to tell this forgotten history, the long-lost backstory to the methods, and the arrogance, of Silicon Valley.Founded in 1959 by some of the nation’s leading social scientists—“the best and the brightest, fatally brilliant, Icaruses with wings of feathers and wax, flying to the sun”—Simulmatics proposed to predict and manipulate the future by way of the computer simulation of human behavior. In summers, with their wives and children in tow, the company’s scientists met on the beach in Long Island under a geodesic, honeycombed dome, where they built a “People Machine” that aimed to model everything from buying a dishwasher to counterinsurgency to casting a vote. Deploying their “People Machine” from New York, Washington, Cambridge, and even Saigon, Simulmatics’ clients included the John F. Kennedy presidential campaign, the New York Times, the Department of Defense, and dozens of major manufacturers: Simulmatics had a hand in everything from political races to the Vietnam War to the Johnson administration’s ill-fated attempt to predict race riots. The company’s collapse was almost as rapid as its ascent, a collapse that involved failed marriages, a suspicious death, and bankruptcy. Exposed for false claims, and even accused of war crimes, it closed its doors in 1970 and all but vanished. Until Lepore came across the records of its remains.The scientists of Simulmatics believed they had invented “the A-bomb of the social sciences.” They did not predict that it would take decades to detonate, like a long-buried grenade. But, in the early years of the twenty-first century, that bomb did detonate, creating a world in which corporations collect data and model behavior and target messages about the most ordinary of decisions, leaving people all over the world, long before the global pandemic, crushed by feelings of helplessness. This history has a past; If Then is its cautionary tale.

    Full of insights, arguments and philosophical perspectives, the book covers an amazing array of topics. Beginning in antiquity with Democritus, it progresses through logic and set theory, computability and complexity theory, quantum computing, cryptography, the information content of quantum states and the interpretation of quantum mechanics. There are also extended discussions about time travel, Newcomb's Paradox, the anthropic principle and the views of Roger Penrose. Aaronson's informal style makes this fascinating book accessible to readers with scientific backgrounds, as well as students and researchers working in physics, computer science, mathematics and philosophy.

    What if everything in life could be reduced to a simple formula? What if numbers were able to tell us which partners we were best matched with – not just in terms of attractiveness, but for a long-term committed marriage? Or if they could say which films would be the biggest hits at the box office, and what changes could be made to those films to make them even more successful? Or even who out of us is likely to commit certain crimes, and when? This may sound like the world of science-fiction, but in fact it is just the tip of the iceberg in a world that is increasingly ruled by complex algorithms and neural networks.In The Formula, Luke Dormehl takes you inside the world of numbers, asking how we came to believe in the all-conquering power of algorithms; introducing the mathematicians, artificial intelligence experts and Silicon Valley entrepreneurs who are shaping this brave new world, and ultimately asking how we survive in an era where numbers can sometimes seem to create as many problems as they solve.