With this practical guide, you’ll learn how to use Python scripts and web APIs to gather and process data from thousands—or even millions—of web pages at once. Ideal for programmers, security professionals, and web administrators familiar with Python, this book not only teaches basic web scraping mechanics, but also delves into more advanced topics, such as analyzing raw data or using scrapers for frontend website testing. Code samples are available to help you understand the concepts in practice. Learn how to parse complicated HTML pages Traverse multiple pages and sites Get a general overview of APIs and how they work Learn several methods for storing the data you scrape Download, read, and extract data from documents Use tools and techniques to clean badly formatted data Read and write natural languages Crawl through forms and logins Understand how to scrape JavaScript Learn image processing and text recognition

    Hundreds of thousands of Python developers around the world rely on Python for general-purpose tasks, Internet scripting, systems programming, user interfaces, and product customization. Available on all major computing platforms, including commercial versions of Unix, Linux, Windows, and Mac OS X, Python is portable, powerful and remarkable easy to use.With its convenient, quick-reference format, "Python Pocket Reference," 3rd Edition is the perfect on-the-job reference. More importantly, it's now been refreshed to cover the language's latest release, Python 2.4. For experienced Python developers, this book is a compact toolbox that delivers need-to-know information at the flip of a page. This third edition also includes an easy-lookup index to help developers find answers fast!Python 2.4 is more than just optimization and library enhancements; it's also chock full of bug fixes and upgrades. And these changes are addressed in the "Python Pocket Reference," 3rd Edition. New language features, new and upgraded built-ins, and new and upgraded modules and packages--they're all clarified in detail.The "Python Pocket Reference," 3rd Edition serves as the perfect companion to "Learning Python" and "Programming Python."

    Discover the “hidden gold” in Python’s standard library and start writing clean and Pythonic code today. Who Should Read This Book: If you’re wondering which lesser known parts in Python you should know about, you’ll get a roadmap with this book. Discover cool (yet practical!) Python tricks and blow your coworkers’ minds in your next code review. If you’ve got experience with legacy versions of Python, the book will get you up to speed with modern patterns and features introduced in Python 3 and backported to Python 2. If you’ve worked with other programming languages and you want to get up to speed with Python, you’ll pick up the idioms and practical tips you need to become a confident and effective Pythonista. If you want to make Python your own and learn how to write clean and Pythonic code, you’ll discover best practices and little-known tricks to round out your knowledge. What Python Developers Say About The Book: "I kept thinking that I wished I had access to a book like this when I started learning Python many years ago." — Mariatta Wijaya, Python Core Developer"This book makes you write better Python code!" — Bob Belderbos, Software Developer at Oracle"Far from being just a shallow collection of snippets, this book will leave the attentive reader with a deeper understanding of the inner workings of Python as well as an appreciation for its beauty." — Ben Felder, Pythonista"It's like having a seasoned tutor explaining, well, tricks!" — Daniel Meyer, Sr. Desktop Administrator at Tesla Inc.

    You can think of the Python programming language as your tool to solve data problems that are beyond the capability of a spreadsheet.Python is an easy to use and easy to learn programming language that is freely available on Macintosh, Windows, or Linux computers. So once you learn Python you can use it for the rest of your career without needing to purchase any software.This book uses the Python 3 language. The earlier Python 2 version of this book is titled "Python for Informatics: Exploring Information".

    LikeWhen does a bee go through metamorphosis?Why does a queen bee sometimes leave her hive?And where does all this honey come from, anyway?!But Nyuki’s biggest question is, “What is this inner voice I hear, and why does it tell me to go forth to adventure?Follow Nyuki on a lifelong journey as she annoys her sisters, avoids predators, and learns to trust her inner voice as she masters the way of the hive.And if you still have questions at the end, the back of the book uncovers even more mysteries about the lives of these incredible insects!

    You'll also learn how to make your programs interactive and how to test your code safely before adding it to a project. In the second half of the book, you'll put your new knowledge into practice with three substantial projects: a Space Invaders-inspired arcade game, data visualizations with Python's super-handy libraries, and a simple web app you can deploy online.As you work through Python Crash Course, you'll learn how to: Use powerful Python libraries and tools, including matplotlib, NumPy, and PygalMake 2D games that respond to keypresses and mouse clicks, and that grow more difficult as the game progressesWork with data to generate interactive visualizationsCreate and customize simple web apps and deploy them safely onlineDeal with mistakes and errors so you can solve your own programming problemsIf you've been thinking seriously about digging into programming, Python Crash Course will get you up to speed and have you writing real programs fast. Why wait any longer? Start your engines and code!

    The algorithms you'll use most often as a programmer have already been discovered, tested, and proven. If you want to take a hard pass on Knuth's brilliant but impenetrable theories and the dense multi-page proofs you'll find in most textbooks, this is the book for you. This fully-illustrated and engaging guide makes it easy for you to learn how to use algorithms effectively in your own programs.Grokking Algorithms is a disarming take on a core computer science topic. In it, you'll learn how to apply common algorithms to the practical problems you face in day-to-day life as a programmer. You'll start with problems like sorting and searching. As you build up your skills in thinking algorithmically, you'll tackle more complex concerns such as data compression or artificial intelligence. Whether you're writing business software, video games, mobile apps, or system utilities, you'll learn algorithmic techniques for solving problems that you thought were out of your grasp. For example, you'll be able to:Write a spell checker using graph algorithmsUnderstand how data compression works using Huffman codingIdentify problems that take too long to solve with naive algorithms, and attack them with algorithms that give you an approximate answer insteadEach carefully-presented example includes helpful diagrams and fully-annotated code samples in Python. By the end of this book, you will know some of the most widely applicable algorithms as well as how and when to use them.

    "More concretely," the authors explain, "it is the controlled manipulation of mathematical formulas, using a collection of techniques for solving problems."

    With the sophisticated algorithms in this book, you can write smart programs to access interesting datasets from other web sites, collect data from users of your own applications, and analyze and understand the data once you've found it.Programming Collective Intelligence takes you into the world of machine learning and statistics, and explains how to draw conclusions about user experience, marketing, personal tastes, and human behavior in general -- all from information that you and others collect every day. Each algorithm is described clearly and concisely with code that can immediately be used on your web site, blog, Wiki, or specialized application. This book explains:Collaborative filtering techniques that enable online retailers to recommend products or media Methods of clustering to detect groups of similar items in a large dataset Search engine features -- crawlers, indexers, query engines, and the PageRank algorithm Optimization algorithms that search millions of possible solutions to a problem and choose the best one Bayesian filtering, used in spam filters for classifying documents based on word types and other features Using decision trees not only to make predictions, but to model the way decisions are made Predicting numerical values rather than classifications to build price models Support vector machines to match people in online dating sites Non-negative matrix factorization to find the independent features in a dataset Evolving intelligence for problem solving -- how a computer develops its skill by improving its own code the more it plays a game Each chapter includes exercises for extending the algorithms to make them more powerful. Go beyond simple database-backed applications and put the wealth of Internet data to work for you. "Bravo! I cannot think of a better way for a developer to first learn these algorithms and methods, nor can I think of a better way for me (an old AI dog) to reinvigorate my knowledge of the details."-- Dan Russell, Google "Toby's book does a great job of breaking down the complex subject matter of machine-learning algorithms into practical, easy-to-understand examples that can be directly applied to analysis of social interaction across the Web today. If I had this book two years ago, it would have saved precious time going down some fruitless paths."-- Tim Wolters, CTO, Collective Intellect

    As the most popular blogging and content management platform available today, WordPress is a powerful tool. This exciting book goes beyond the basics and delves into the heart of the WordPress system, offering overviews of the functional aspects of WordPress as well as plug-in and theme development. What is covered in this book?WordPress as a Content Management System Hosting Options Installing WordPress Files Database Configuration Dashboard Widgets Customizing the Dashboard Creating and Managing Content Categorizing Your Content Working with Media Comments and Discussion Working with Users Managing, Adding, Upgrading, and Using the Theme Editor Working with Widgets Adding and Managing New Plugins Configuring WordPress Exploring the Code Configuring Key Files wp-config.php file Advanced wp-config Options What's in the Core? WordPress Codex and Resources Understanding and customizing the Loop Building A Custom Query Complex Database Operations Dealing With Errors Direct Database Manipulation Building Your Own Taxonomies Plugin Packaging Create a Dashboard Widget Creating a Plugin Example Publish to the Plugin Directory Installing a Theme Creating Your Own Theme How and When to Use Custom Page Templates How to Use Custom Page Templates Pushing Content from WordPress to Other Sites Usability and Usability Testing Getting Your Site Found How Web Standards Get Your Data Discovered Load Balancing Your WordPress Site Securing Your WordPress Site Using WordPress in the Enterprise Is WordPress Right for Your Enterprise? and much more!

    In 1877 he left home in search of an education, eventually earning a master's degree. In 1896, Booker T. Washington invited Carver to start the agricultural department at the all-black-staffed Tuskegee Institute, where he spent the rest of his life seeking solutions to the poverty among landless black farmers by developing new uses for soil-replenishing crops such as peanuts, cowpeas, and sweet potatoes. Carver's achievements as a botanist and inventor were balanced by his gifts as a painter, musician, and teacher. This Newbery Honor Book and Coretta Scott King Author Honor Book by Marilyn Nelson provides a compelling and revealing portrait of Carver's complex, richly interior, profoundly devout life.

    Until now. This practical guide provides the first comprehensive overview of software architecture's many aspects. You'll examine architectural characteristics, architectural patterns, component determination, diagramming and presenting architecture, evolutionary architecture, and many other topics.Authors Neal Ford and Mark Richards help you learn through examples in a variety of popular programming languages, such as Java, C#, JavaScript, and others. You'll focus on architecture principles with examples that apply across all technology stacks.

    You'll be guided through the entire process of creating a gorgeous, usable web site by applying the timeless principles of user-centered design.Even if you're short on design skills, with this book you'll be creating your own stunning web sites in no time at all.Throughout, the focus is on simple and practical techniques that anyone can use - you don't need to have gone to art school or have artistic flair to create stunning designs using the methods outlined in this book.The book's full-color layout and large format (8" x 10") make Sexy Web Design a pleasure to read.Master key web interface design principles Design amazing web interfaces from scratch Create beautiful, yet functional, web sites Unleash your artistic talents And much more Who should read this book? Whether you're completely new to web design, a seasoned pro looking for inspiration, or a developer wanting to improve your sites' aesthetics, there's something for everyone here.How? Because instead of trying to cover every possible area of creating a web site, we've focused purely on the design stage; that is, everything that happens before a single line of code is written.However, great design is more than just aesthetics. Long before we open our graphics program of choice, we'll be conducting research, dealing with clients, responding to briefs, sketching out sitemaps, planning information architecture, moving from doodles to diagrams, exploring different ways of interactivity, and building upon design traditions.But ultimately, you'll be finding out how to create web sites that look drop-dead gorgeous.

    Consumer's identities are being stolen, and a person's every step is being tracked and stored. What once might have been dismissed as paranoia is now a hard truth, and privacy is a luxury few can afford or understand.In this explosive yet practical book, Kevin Mitnick illustrates what is happening without your knowledge--and he teaches you "the art of invisibility." Mitnick is the world's most famous--and formerly the Most Wanted--computer hacker. He has hacked into some of the country's most powerful and seemingly impenetrable agencies and companies, and at one point he was on a three-year run from the FBI. Now, though, Mitnick is reformed and is widely regarded as the expert on the subject of computer security. He knows exactly how vulnerabilities can be exploited and just what to do to prevent that from happening. In THE ART OF INVISIBILITY Mitnick provides both online and real life tactics and inexpensive methods to protect you and your family, in easy step-by-step instructions. He even talks about more advanced "elite" techniques, which, if used properly, can maximize your privacy. Invisibility isn't just for superheroes--privacy is a power you deserve and need in this modern age.

    The authors explain each category of vulnerability using real-world examples, screen shots and code extracts. The book is extremely practical in focus, and describes in detail the steps involved in detecting and exploiting each kind of security weakness found within a variety of applications such as online banking, e-commerce and other web applications. The topics covered include bypassing login mechanisms, injecting code, exploiting logic flaws and compromising other users. Because every web application is different, attacking them entails bringing to bear various general principles, techniques and experience in an imaginative way. The most successful hackers go beyond this, and find ways to automate their bespoke attacks. This handbook describes a proven methodology that combines the virtues of human intelligence and computerized brute force, often with devastating results.The authors are professional penetration testers who have been involved in web application security for nearly a decade. They have presented training courses at the Black Hat security conferences throughout the world. Under the alias "PortSwigger," Dafydd developed the popular Burp Suite of web application hack tools.