With this practical guide, you’ll learn how to use Python scripts and web APIs to gather and process data from thousands—or even millions—of web pages at once. Ideal for programmers, security professionals, and web administrators familiar with Python, this book not only teaches basic web scraping mechanics, but also delves into more advanced topics, such as analyzing raw data or using scrapers for frontend website testing. Code samples are available to help you understand the concepts in practice. Learn how to parse complicated HTML pages Traverse multiple pages and sites Get a general overview of APIs and how they work Learn several methods for storing the data you scrape Download, read, and extract data from documents Use tools and techniques to clean badly formatted data Read and write natural languages Crawl through forms and logins Understand how to scrape JavaScript Learn image processing and text recognition

    Difficult issues need to be figured out, such as scalability, consistency, reliability, efficiency, and maintainability. In addition, we have an overwhelming variety of tools, including relational databases, NoSQL datastores, stream or batch processors, and message brokers. What are the right choices for your application? How do you make sense of all these buzzwords?In this practical and comprehensive guide, author Martin Kleppmann helps you navigate this diverse landscape by examining the pros and cons of various technologies for processing and storing data. Software keeps changing, but the fundamental principles remain the same. With this book, software engineers and architects will learn how to apply those ideas in practice, and how to make full use of data in modern applications. Peer under the hood of the systems you already use, and learn how to use and operate them more effectively Make informed decisions by identifying the strengths and weaknesses of different tools Navigate the trade-offs around consistency, scalability, fault tolerance, and complexity Understand the distributed systems research upon which modern databases are built Peek behind the scenes of major online services, and learn from their architectures

    Discover the “hidden gold” in Python’s standard library and start writing clean and Pythonic code today. Who Should Read This Book: If you’re wondering which lesser known parts in Python you should know about, you’ll get a roadmap with this book. Discover cool (yet practical!) Python tricks and blow your coworkers’ minds in your next code review. If you’ve got experience with legacy versions of Python, the book will get you up to speed with modern patterns and features introduced in Python 3 and backported to Python 2. If you’ve worked with other programming languages and you want to get up to speed with Python, you’ll pick up the idioms and practical tips you need to become a confident and effective Pythonista. If you want to make Python your own and learn how to write clean and Pythonic code, you’ll discover best practices and little-known tricks to round out your knowledge. What Python Developers Say About The Book: "I kept thinking that I wished I had access to a book like this when I started learning Python many years ago." — Mariatta Wijaya, Python Core Developer"This book makes you write better Python code!" — Bob Belderbos, Software Developer at Oracle"Far from being just a shallow collection of snippets, this book will leave the attentive reader with a deeper understanding of the inner workings of Python as well as an appreciation for its beauty." — Ben Felder, Pythonista"It's like having a seasoned tutor explaining, well, tricks!" — Daniel Meyer, Sr. Desktop Administrator at Tesla Inc.

    In this one-of-a-kind text, author V. Anton Spraul breaks down the ways that programmers solve problems and teaches you what other introductory books often ignore: how to Think Like a Programmer. Each chapter tackles a single programming concept, like classes, pointers, and recursion, and open-ended exercises throughout challenge you to apply your knowledge. You'll also learn how to:Split problems into discrete components to make them easier to solve Make the most of code reuse with functions, classes, and libraries Pick the perfect data structure for a particular job Master more advanced programming tools like recursion and dynamic memory Organize your thoughts and develop strategies to tackle particular types of problems Although the book's examples are written in C++, the creative problem-solving concepts they illustrate go beyond any particular language; in fact, they often reach outside the realm of computer science. As the most skillful programmers know, writing great code is a creative art—and the first step in creating your masterpiece is learning to Think Like a Programmer.

    If you're new to Elixir, its functional style can seem difficult, but with help from this hands-on introduction, you'll scale the learning curve and discover how enjoyable, powerful, and fun this language can be. Elixir combines the robust functional programming of Erlang with an approach that looks more like Ruby and reaches toward metaprogramming with powerful macro features.Authors Simon St. Laurent and J. David Eisenberg show you how to write simple Elixir programs by teaching you one skill at a time. You’ll learn about pattern matching, recursion, message passing, process-oriented programming, and establishing pathways for data rather than telling it where to go. By the end of your journey, you’ll understand why Elixir is ideal for concurrency and resilience.* Get comfortable with IEx, Elixir's command line interface* Become familiar with Elixir’s basic structures by working with numbers* Discover atoms, pattern matching, and guards: the foundations of your program structure* Delve into the heart of Elixir processing with recursion, strings, lists, and higher-order functions* Create processes, send messages among them, and apply pattern matching to incoming messages* Store and manipulate structured data with Erlang Term * Storage (ETS) and the Mnesia database* Build resilient applications with the Open Telecom Platform (OTP)* Define macros with Elixir's meta-programming tools.

    This long-awaited revision contains changes throughout the text. There are new implementations of most of the major programming systems in the book, including the interpreters and compilers, and the authors have incorporated many small changes that reflect their experience teaching the course at MIT since the first edition was published. A new theme has been introduced that emphasizes the central role played by different approaches to dealing with time in computational models: objects with state, concurrent programming, functional programming and lazy evaluation, and nondeterministic programming. There are new example sections on higher-order procedures in graphics and on applications of stream processing in numerical programming, and many new exercises. In addition, all the programs have been reworked to run in any Scheme implementation that adheres to the IEEE standard.

     They are also used in countless industries such as medicine, manufacturing and finance to help companies make better decisions and reduce risk. Whether coded or scratched out by hand, both algorithms are powerful tools that can make a significant impact. This book is a visual introduction for beginners that unpacks the fundamentals of decision trees and random forests. If you want to dig into the basics with a visual twist plus create your own machine learning algorithms in Python, this book is for you.

    Today, that taboo is dead. The causal revolution, instigated by Judea Pearl and his colleagues, has cut through a century of confusion and established causality -- the study of cause and effect -- on a firm scientific basis. His work explains how we can know easy things, like whether it was rain or a sprinkler that made a sidewalk wet; and how to answer hard questions, like whether a drug cured an illness. Pearl's work enables us to know not just whether one thing causes another: it lets us explore the world that is and the worlds that could have been. It shows us the essence of human thought and key to artificial intelligence. Anyone who wants to understand either needs The Book of Why.

    By reconstructing authentic, complex AI programs using state-of-the-art Common Lisp, the book teaches students and professionals how to build and debug robust practical programs, while demonstrating superior programming style and important AI concepts. The author strongly emphasizes the practical performance issues involved in writing real working programs of significant size. Chapters on troubleshooting and efficiency are included, along with a discussion of the fundamentals of object-oriented programming and a description of the main CLOS functions. This volume is an excellent text for a course on AI programming, a useful supplement for general AI courses and an indispensable reference for the professional programmer.

    In this cookbook, prolific hacker and author Simon Monk provides more than 200 practical recipes for running this tiny low-cost computer with Linux, programming it with Python, and hooking up sensors, motors, and other hardware—including Arduino.You’ll also learn basic principles to help you use new technologies with Raspberry Pi as its ecosystem develops. Python and other code examples from the book are available on GitHub. This cookbook is ideal for programmers and hobbyists familiar with the Pi through resources such as Getting Started with Raspberry Pi (O’Reilly).Set up and manage your Raspberry PiConnect the Pi to a networkWork with its Linux-based operating systemUse the Pi’s ready-made softwareProgram Raspberry Pi with PythonControl hardware through the GPIO connectorUse Raspberry Pi to run different types of motorsWork with switches, keypads, and other digital inputsHook up sensors for taking various measurementsAttach different displays, such as an LED matrixCreate dynamic projects with Raspberry Pi and Arduino Make sure to check out 10 of the over 60 video recipes for this book at: http://razzpisampler.oreilly.com/ You can purchase all recipes at:

    Cryptography is vital to keeping information safe, in an era when the formula to do so becomes more and more challenging. Written by a team of world-renowned cryptography experts, this essential guide is the definitive introduction to all major areas of cryptography: message security, key negotiation, and key management. You'll learn how to think like a cryptographer. You'll discover techniques for building cryptography into products from the start and you'll examine the many technical changes in the field.After a basic overview of cryptography and what it means today, this indispensable resource covers such topics as block ciphers, block modes, hash functions, encryption modes, message authentication codes, implementation issues, negotiation protocols, and more. Helpful examples and hands-on exercises enhance your understanding of the multi-faceted field of cryptography.An author team of internationally recognized cryptography experts updates you on vital topics in the field of cryptography Shows you how to build cryptography into products from the start Examines updates and changes to cryptography Includes coverage on key servers, message security, authentication codes, new standards, block ciphers, message authentication codes, and more Cryptography Engineering gets you up to speed in the ever-evolving field of cryptography.

    Increasingly, scientific breakthroughs will be powered by advanced computing capabilities that help researchers manipulate and explore massive datasets. The speed at which any given scientific discipline advances will depend on how well its researchers collaborate with one another, and with technologists, in areas of eScience such as databases, workflow management, visualization, and cloud-computing technologies. This collection of essays expands on the vision of pioneering computer scientist Jim Gray for a new, fourth paradigm of discovery based on data-intensive science and offers insights into how it can be fully realized.

    "Monitoring Monitoring" explains what makes your monitoring less than stellar, and provides a practical approach to designing and implementing a monitoring strategy, from the application down to the hardware in the datacenter and everything in between.In the world of technical operations, monitoring is core to everything you do. In today s changing landscape of microservices, cloud infrastructure, and more, monitoring is experiencing a new surge of growth, bringing along new methodologies, new ways of thinking, and new tools.Complete with a primer on statistics and a monitoring vocabulary, this book helps you identify the main areas you need to monitor and shows you how to approach them. It s ideal for operations engineers, system administrators, system and software engineers, site reliability engineers, network engineers, and other operations professionals."

    You'd like to learn how to do more work with SQL inside the database before pushing data across the network to your applications. You'd like to take your SQL skills to the next level.Let's face it, SQL is a deceptively simple language to learn, and many database developers never go far beyond the simple statement: SELECT columns FROM table WHERE conditions. But there is so much more you can do with the language. In the SQL Cookbook, experienced SQL developer Anthony Molinaro shares his favorite SQL techniques and features. You'll learn about:Window functions, arguably the most significant enhancement to SQL in the past decade. If you're not using these, you're missing outPowerful, database-specific features such as SQL Server's PIVOT and UNPIVOT operators, Oracle's MODEL clause, and PostgreSQL's very useful GENERATE_SERIES functionPivoting rows into columns, reverse-pivoting columns into rows, using pivoting to facilitate inter-row calculations, and double-pivoting a result setBucketization, and why you should never use that term in Brooklyn.How to create histograms, summarize data into buckets, perform aggregations over a moving range of values, generate running-totals and subtotals, and other advanced, data warehousing techniquesThe technique of walking a string, which allows you to use SQL to parse through the characters, words, or delimited elements of a stringWritten in O'Reilly's popular Problem/Solution/Discussion style, the SQL Cookbook is sure to please. Anthony's credo is: When it comes down to it, we all go to work, we all have bills to pay, and we all want to go home at a reasonable time and enjoy what's still available of our days. The SQL Cookbook moves quickly from problem to solution, saving you time each step of the way.

    But in spirit, the title is apt, as the book does cover a much broader range of topics than a typical introductory book on mathematical statistics. This book is for people who want to learn probability and statistics quickly. It is suitable for graduate or advanced undergraduate students in computer science, mathematics, statistics, and related disciplines. The book includes modern topics like nonparametric curve estimation, bootstrapping, and clas- sification, topics that are usually relegated to follow-up courses. The reader is presumed to know calculus and a little linear algebra. No previous knowledge of probability and statistics is required. Statistics, data mining, and machine learning are all concerned with collecting and analyzing data. For some time, statistics research was con- ducted in statistics departments while data mining and machine learning re- search was conducted in computer science departments. Statisticians thought that computer scientists were reinventing the wheel. Computer scientists thought that statistical theory didn't apply to their problems. Things are changing. Statisticians now recognize that computer scientists are making novel contributions while computer scientists now recognize the generality of statistical theory and methodology. Clever data mining algo- rithms are more scalable than statisticians ever thought possible. Formal sta- tistical theory is more pervasive than computer scientists had realized.