With the sophisticated algorithms in this book, you can write smart programs to access interesting datasets from other web sites, collect data from users of your own applications, and analyze and understand the data once you've found it.Programming Collective Intelligence takes you into the world of machine learning and statistics, and explains how to draw conclusions about user experience, marketing, personal tastes, and human behavior in general -- all from information that you and others collect every day. Each algorithm is described clearly and concisely with code that can immediately be used on your web site, blog, Wiki, or specialized application. This book explains:Collaborative filtering techniques that enable online retailers to recommend products or media Methods of clustering to detect groups of similar items in a large dataset Search engine features -- crawlers, indexers, query engines, and the PageRank algorithm Optimization algorithms that search millions of possible solutions to a problem and choose the best one Bayesian filtering, used in spam filters for classifying documents based on word types and other features Using decision trees not only to make predictions, but to model the way decisions are made Predicting numerical values rather than classifications to build price models Support vector machines to match people in online dating sites Non-negative matrix factorization to find the independent features in a dataset Evolving intelligence for problem solving -- how a computer develops its skill by improving its own code the more it plays a game Each chapter includes exercises for extending the algorithms to make them more powerful. Go beyond simple database-backed applications and put the wealth of Internet data to work for you. "Bravo! I cannot think of a better way for a developer to first learn these algorithms and methods, nor can I think of a better way for me (an old AI dog) to reinvigorate my knowledge of the details."-- Dan Russell, Google "Toby's book does a great job of breaking down the complex subject matter of machine-learning algorithms into practical, easy-to-understand examples that can be directly applied to analysis of social interaction across the Web today. If I had this book two years ago, it would have saved precious time going down some fruitless paths."-- Tim Wolters, CTO, Collective Intellect

    

    

    Dyson traces the course of the information revolution, illuminating the lives and work of visionaries - from the time of Thomas Hobbes to the time of John von Neumann - who foresaw the development of artificial intelligence, artificial life, and artificial mind. This book derives both its title and its outlook from Samuel Butler's 1863 essay "Darwin Among the Machines." Observing the beginnings of miniaturization, self-reproduction, and telecommunication among machines, Butler predicted that nature's intelligence, only temporarily subservient to technology, would resurface to claim our creations as her own. Weaving a cohesive narrative among his brilliant predecessors, Dyson constructs a straightforward, convincing, and occasionally frightening view of the evolution of mind in the global network, on a level transcending our own. Dyson concludes that we are in the midst of an experiment that echoes the prehistory of human intelligence and the origins of life. Just as the exchange of coded molecular instructions brought life as we know it to the early earth's primordial soup, and as language and mind combined to form the culture in which we live, so, in the digital universe, are computer programs and worldwide networks combining to produce an evolutionary theater in which the distinctions between nature and technology are increasingly obscured. Nature, believes Dyson, is on the side of the machines.

    This foundational text is the first comprehensive introduction to statistical natural language processing (NLP) to appear. The book contains all the theory and algorithms needed for building NLP tools. It provides broad but rigorous coverage of mathematical and linguistic foundations, as well as detailed discussion of statistical methods, allowing students and researchers to construct their own implementations. The book covers collocation finding, word sense disambiguation, probabilistic parsing, information retrieval, and other applications.

    But what if you could have your computer do them for you?In "Automate the Boring Stuff with Python," you'll learn how to use Python to write programs that do in minutes what would take you hours to do by hand no prior programming experience required. Once you've mastered the basics of programming, you'll create Python programs that effortlessly perform useful and impressive feats of automation to: Search for text in a file or across multiple filesCreate, update, move, and rename files and foldersSearch the Web and download online contentUpdate and format data in Excel spreadsheets of any sizeSplit, merge, watermark, and encrypt PDFsSend reminder emails and text notificationsFill out online formsStep-by-step instructions walk you through each program, and practice projects at the end of each chapter challenge you to improve those programs and use your newfound skills to automate similar tasks.Don't spend your time doing work a well-trained monkey could do. Even if you've never written a line of code, you can make your computer do the grunt work. Learn how in "Automate the Boring Stuff with Python.""

    This concise introduction shows you how to perform statistical analysis computationally, rather than mathematically, with programs written in Python.You'll work with a case study throughout the book to help you learn the entire data analysis process—from collecting data and generating statistics to identifying patterns and testing hypotheses. Along the way, you'll become familiar with distributions, the rules of probability, visualization, and many other tools and concepts.Develop your understanding of probability and statistics by writing and testing codeRun experiments to test statistical behavior, such as generating samples from several distributionsUse simulations to understand concepts that are hard to grasp mathematicallyLearn topics not usually covered in an introductory course, such as Bayesian estimationImport data from almost any source using Python, rather than be limited to data that has been cleaned and formatted for statistics toolsUse statistical inference to answer questions about real-world data

    Acclaimed technologist and inventor Amir Husain explains how we can live amidst the coming age of sentient machines and artificial intelligence—and not only survive, but thrive.Artificial “machine” intelligence is playing an ever-greater role in our society. We are already using cruise control in our cars, automatic checkout at the drugstore, and are unable to live without our smartphones. The discussion around AI is polarized; people think either machines will solve all problems for everyone, or they will lead us down a dark, dystopian path into total human irrelevance. Regardless of what you believe, the idea that we might bring forth intelligent creation can be intrinsically frightening. But what if our greatest role as humans so far is that of creators? Amir Husain, a brilliant inventor and computer scientist, argues that we are on the cusp of writing our next, and greatest, creation myth. It is the dawn of a new form of intellectual diversity, one that we need to embrace in order to advance the state of the art in many critical fields, including security, resource management, finance, and energy. “In The Sentient Machine, Husain prepares us for a brighter future; not with hyperbole about right and wrong, but with serious arguments about risk and potential” (Dr. Greg Hyslop, Chief Technology Officer, The Boeing Company). He addresses broad existential questions surrounding the coming of AI: Why are we valuable? What can we create in this world? How are we intelligent? What constitutes progress for us? And how might we fail to progress? Husain boils down complex computer science and AI concepts into clear, plainspoken language and draws from a wide variety of cultural and historical references to illustrate his points. Ultimately, Husain challenges many of our societal norms and upends assumptions we hold about “the good life.”

    This book focuses on practical algorithms that have been used to solve key problems in data mining and which can be used on even the largest datasets. It begins with a discussion of the map-reduce framework, an important tool for parallelizing algorithms automatically. The authors explain the tricks of locality-sensitive hashing and stream processing algorithms for mining data that arrives too fast for exhaustive processing. The PageRank idea and related tricks for organizing the Web are covered next. Other chapters cover the problems of finding frequent itemsets and clustering. The final chapters cover two applications: recommendation systems and Web advertising, each vital in e-commerce. Written by two authorities in database and Web technologies, this book is essential reading for students and practitioners alike.

     As someone that has used the Bash shell almost daily for over 15 years, I've accumulated several command line "tricks" that have saved me time and frustration. Bash Command Line Pro Tips is a collection of 10 techniques that you can put to use right away to increase your efficiency at the command line. Here is what you will learn by reading Bash Command Line Pro Tips: Tip 1: Tab Completion Tip 2: Change to the Previous Directory Tip 3: Reuse the Last Item from the Previous Command Line Tip 4: Rerun a Command That Starts with a given String Tip 5: Command Substitution Tip 6: Use a for Loop at the Command Line Tip 7: Rerun the Previous Command with Root Privileges Tip 8: Rerun the Previous Command While Substituting a String Tip 9: Reuse a Word on the Same Command Line Tip 10: Fix Typos and Shorten Lengthy Commands with Aliases Scroll up, click the "Buy Now With 1-Click" button to start leaning these powerful Linux Command Line Tips.

    However, most discussions of Bayesian inference rely on intensely complex mathematical analyses and artificial examples, making it inaccessible to anyone without a strong mathematical background. Now, though, Cameron Davidson-Pilon introduces Bayesian inference from a computational perspective, bridging theory to practice-freeing you to get results using computing power. Bayesian Methods for Hackers illuminates Bayesian inference through probabilistic programming with the powerful PyMC language and the closely related Python tools NumPy, SciPy, and Matplotlib. Using this approach, you can reach effective solutions in small increments, without extensive mathematical intervention. Davidson-Pilon begins by introducing the concepts underlying Bayesian inference, comparing it with other techniques and guiding you through building and training your first Bayesian model. Next, he introduces PyMC through a series of detailed examples and intuitive explanations that have been refined after extensive user feedback. You'll learn how to use the Markov Chain Monte Carlo algorithm, choose appropriate sample sizes and priors, work with loss functions, and apply Bayesian inference in domains ranging from finance to marketing. Once you've mastered these techniques, you'll constantly turn to this guide for the working PyMC code you need to jumpstart future projects. Coverage includes - Learning the Bayesian "state of mind" and its practical implications - Understanding how computers perform Bayesian inference - Using the PyMC Python library to program Bayesian analyses - Building and debugging models with PyMC - Testing your model's "goodness of fit" - Opening the "black box" of the Markov Chain Monte Carlo algorithm to see how and why it works - Leveraging the power of the "Law of Large Numbers" - Mastering key concepts, such as clustering, convergence, autocorrelation, and thinning - Using loss functions to measure an estimate's weaknesses based on your goals and desired outcomes - Selecting appropriate priors and understanding how their influence changes with dataset size - Overcoming the "exploration versus exploitation" dilemma: deciding when "pretty good" is good enough - Using Bayesian inference to improve A/B testing - Solving data science problems when only small amounts of data are available Cameron Davidson-Pilon has worked in many areas of applied mathematics, from the evolutionary dynamics of genes and diseases to stochastic modeling of financial prices. His contributions to the open source community include lifelines, an implementation of survival analysis in Python. Educated at the University of Waterloo and at the Independent University of Moscow, he currently works with the online commerce leader Shopify.

    'Big data', 'data science', and 'machine learning' have become familiar terms in the news, as statistical methods are brought to bear upon the enormous data sets of modern science and commerce. How did we get here? And where are we going? This book takes us on an exhilarating journey through the revolution in data analysis following the introduction of electronic computation in the 1950s. Beginning with classical inferential theories - Bayesian, frequentist, Fisherian - individual chapters take up a series of influential topics: survival analysis, logistic regression, empirical Bayes, the jackknife and bootstrap, random forests, neural networks, Markov chain Monte Carlo, inference after model selection, and dozens more. The distinctly modern approach integrates methodology and algorithms with statistical inference. The book ends with speculation on the future direction of statistics and data science.

    Introducing students to the many facets of neural networks, this text provides many case studies to illustrate their real-life, practical applications.

    Using sample charts, graphics, case studies, and war stories, Yankee Group Security Expert Andrew Jaquith demonstrates exactly how to establish effective metrics based on your organization's unique requirements. You'll discover how to quantify hard-to-measure security activities, compile and analyze all relevant data, identify strengths and weaknesses, set cost-effective priorities for improvement, and craft compelling messages for senior management. Security Metrics successfully bridges management's quantitative viewpoint with the nuts-and-bolts approach typically taken by security professionals. It brings together expert solutions drawn from Jaquith's extensive consulting work in the software, aerospace, and financial services industries, including new metrics presented nowhere else. You'll learn how to: - Replace nonstop crisis response with a systematic approach to security improvement - Understand the differences between "good" and "bad" metrics - Measure coverage and control, vulnerability management, password quality, patch latency, benchmark scoring, and business-adjusted risk - Quantify the effectiveness of security acquisition, implementation, and other program activities - Organize, aggregate, and analyze your data to bring out key insights - Use visualization to understand and communicate security issues more clearly - Capture valuable data from firewalls and antivirus logs, third-party auditor reports, and other resources - Implement balanced scorecards that present compact, holistic views of organizational security effectiveness Whether you're an engineer or consultant responsible for security and reporting to management-or an executive who needs better information for decision-making-Security Metrics is the resource you have been searching for. Andrew Jaquith, program manager for Yankee Group's Security Solutions and Services Decision Service, advises enterprise clients on prioritizing and managing security resources. He also helps security vendors develop product, service, and go-to-market strategies for reaching enterprise customers. He co-founded @stake, Inc., a security consulting pioneer acquired by Symantec Corporation in 2004. His application security and metrics research has been featured in CIO, CSO, InformationWeek, IEEE Security and Privacy, and The Economist. Foreword Preface Acknowledgments About the Author Chapter 1 Introduction: Escaping the Hamster Wheel of Pain Chapter 2 Defining Security Metrics Chapter 3 Diagnosing Problems and Measuring Technical Security Chapter 4 Measuring Program Effectiveness Chapter 5 Analysis Techniques Chapter 6 Visualization Chapter 7 Automating Metrics Calculations Chapter 8 Designing Security Scorecards Index

    This text concisely describes the Python language and its programming environment for those readers already familiar with languages such as C and C++.