With this practical guide, you’ll learn how to use Python scripts and web APIs to gather and process data from thousands—or even millions—of web pages at once. Ideal for programmers, security professionals, and web administrators familiar with Python, this book not only teaches basic web scraping mechanics, but also delves into more advanced topics, such as analyzing raw data or using scrapers for frontend website testing. Code samples are available to help you understand the concepts in practice. Learn how to parse complicated HTML pages Traverse multiple pages and sites Get a general overview of APIs and how they work Learn several methods for storing the data you scrape Download, read, and extract data from documents Use tools and techniques to clean badly formatted data Read and write natural languages Crawl through forms and logins Understand how to scrape JavaScript Learn image processing and text recognition

    Discover the “hidden gold” in Python’s standard library and start writing clean and Pythonic code today. Who Should Read This Book: If you’re wondering which lesser known parts in Python you should know about, you’ll get a roadmap with this book. Discover cool (yet practical!) Python tricks and blow your coworkers’ minds in your next code review. If you’ve got experience with legacy versions of Python, the book will get you up to speed with modern patterns and features introduced in Python 3 and backported to Python 2. If you’ve worked with other programming languages and you want to get up to speed with Python, you’ll pick up the idioms and practical tips you need to become a confident and effective Pythonista. If you want to make Python your own and learn how to write clean and Pythonic code, you’ll discover best practices and little-known tricks to round out your knowledge. What Python Developers Say About The Book: "I kept thinking that I wished I had access to a book like this when I started learning Python many years ago." — Mariatta Wijaya, Python Core Developer"This book makes you write better Python code!" — Bob Belderbos, Software Developer at Oracle"Far from being just a shallow collection of snippets, this book will leave the attentive reader with a deeper understanding of the inner workings of Python as well as an appreciation for its beauty." — Ben Felder, Pythonista"It's like having a seasoned tutor explaining, well, tricks!" — Daniel Meyer, Sr. Desktop Administrator at Tesla Inc.

    Widely accepted as the authoritative text and reference on electronic circuit design, both analog and digital, this book revolutionized the teaching of electronics by emphasizing the methods actually used by circuit designers -- a combination of some basic laws, rules of thumb, and a large bag of tricks. The result is a largely nonmathematical treatment that encourages circuit intuition, brainstorming, and simplified calculations of circuit values and performance. The new Art of Electronics retains the feeling of informality and easy access that helped make the first edition so successful and popular. It is an ideal first textbook on electronics for scientists and engineers and an indispensable reference for anyone, professional or amateur, who works with electronic circuits.

    This full color book teaches you fundamental programming and robotics skills as you build and program powerful robots that move, monitor sensors, and respond to their environments. You will build seven increasingly sophisticated robots like the BRICK SORT3R (a robot that sorts bricks by color and size), the THE SNATCH3R (an autonomous robotic arm), and LAVA R3X (a humanoid that walks and talks). Valk also covers more advanced programming techniques like data wires and variables, and shows you how to create custom remote control programs. More than 100 building and programming challenges throughout encourage you to think creatively and to apply what you've learned to invent your own robots.

    It has become popular in recent years because of its simplicity, conciseness, and clarity. This book teaches functional programming as a way of thinking and problem solving, using Haskell, the most popular purely functional language. Rather than using the conventional (boring) mathematical examples commonly found in other programming language textbooks, the author uses examples drawn from multimedia applications, including graphics, animation, and computer music, thus rewarding the reader with working programs for inherently more interesting applications. Aimed at both beginning and advanced programmers, this tutorial begins with a gentle introduction to functional programming and moves rapidly on to more advanced topics. Details about progamming in Haskell are presented in boxes throughout the text so they can be easily found and referred to.

    The concepts and techniques described are the actual ones used by real game studios like Electronic Arts and Naughty Dog. The examples are often grounded in specific technologies, but the discussion extends way beyond any particular engine or API. The references and citations make it a great jumping off point for those who wish to dig deeper into any particular aspect of the game development process.Intended as the text for a college level series in game programming, this book can also be used by amateur software engineers, hobbyists, self-taught game programmers, and existing members of the game industry. Junior game engineers can use it to solidify their understanding of game technology and engine architecture. Even senior engineers who specialize in one particular field of game development can benefit from the bigger picture presented in these pages.

    It details the core protocols that make TCP/IP internetworks function, and the most important classical TCP/IP applications. Its personal, easy-going writing style lets anyone understand the dozens of protocols and technologies that run the Internet, with full coverage of PPP, ARP, IP, IPv6, IP NAT, IPSec, Mobile IP, ICMP, RIP, BGP, TCP, UDP, DNS, DHCP, SNMP, FTP, SMTP, NNTP, HTTP, Telnet and much more. The author offers not only a detailed view of the TCP/IP protocol suite, but also describes networking fundamentals and the important OSI Reference Model.

    You'll learn how to create responsive, data-driven websites with PHP, MySQL, and JavaScript, regardless of whether you already know how to program. Discover how the powerful combination of PHP and MySQL provides an easy way to build modern websites complete with dynamic data and user interaction. You'll also learn how to add JavaScript to create rich Internet applications and websites.Learning PHP, MySQL, and JavaScript explains each technology separately, shows you how to combine them, and introduces valuable web programming concepts, including objects, XHTML, cookies, and session management. You'll practice what you've learned with review questions in each chapter, and find a sample social networking platform built with the elements introduced in this book. This book will help you:-Understand PHP essentials and the basics of object-oriented programming-Master MySQL, from database structure to complex queries-Create web pages with PHP and MySQL by integrating forms and other HTML features-Learn about JavaScript, from functions and event handling to accessing the Document Object Model-Use libraries and packages, including the Smarty web template system, PEAR program repository, and the Yahoo! User Interface Library -Make Ajax calls and turn your website into a highly dynamic environment-Upload and manipulate files and images, validate user input, and secure your applications

    What sets this book apart is its focus on learning. The authors have made the content of OOAD accessible, usable for the practitioner." Ivar Jacobson, Ivar Jacobson Consulting"I just finished reading HF OOA&D and I loved it! The thing I liked most about this book was its focus on why we do OOA&D-to write great software!" Kyle Brown, Distinguished Engineer, IBM"Hidden behind the funny pictures and crazy fonts is a serious, intelligent, extremely well-crafted presentation of OO Analysis and Design. As I read the book, I felt like I was looking over the shoulder of an expert designer who was explaining to me what issues were important at each step, and why." Edward Sciore, Associate Professor, Computer Science Department, Boston College Tired of reading Object Oriented Analysis and Design books that only makes sense after you're an expert? You've heard OOA&D can help you write great software every time-software that makes your boss happy, your customers satisfied and gives you more time to do what makes you happy.But how?Head First Object-Oriented Analysis & Design shows you how to analyze, design, and write serious object-oriented software: software that's easy to reuse, maintain, and extend; software that doesn't hurt your head; software that lets you add new features without breaking the old ones. Inside you will learn how to:Use OO principles like encapsulation and delegation to build applications that are flexible Apply the Open-Closed Principle (OCP) and the Single Responsibility Principle (SRP) to promote reuse of your code Leverage the power of design patterns to solve your problems more efficiently Use UML, use cases, and diagrams to ensure that all stakeholders are communicating clearly to help you deliver the right software that meets everyone's needs.By exploiting how your brain works, Head First Object-Oriented Analysis & Design compresses the time it takes to learn and retain complex information. Expect to have fun, expect to learn, expect to be writing great software consistently by the time you're finished reading this!

    The commercial success of a game is often dependent upon the quality of the AI, yet the engineering of AI is often begun late in the development process and is frequently misunderstood. In this book, Ian Millington brings extensive professional experience to the problem of improving the quality of AI in games. A game developer since 1987, he was founder of Mindlathe Ltd., at the time the largest specialist AI company in gaming. Ian shows how to think about AI as an integral part of game play. He describes numerous examples from real games and explores the underlying ideas through detailed case studies. He goes further to introduce many techniques little used by developers today. The book's CD-ROM contains a library of C++ source code and demonstration programs, and provides access to a website with a complete commercial source code library of AI algorithms and techniques. * A comprehensive, professional tutorial and reference to implement true AI in games.* Walks through the entire development process from beginning to end.* Includes over 100 pseudo code examples of techniques used in commercial games, case studies for all major genres, a CD-ROM and companion website with extensive C++ source code implementations for Windows, and source code libraries for Linux and OS X available through the website.

    And it’s still short, profusely illustrated…and best of all–fun to read.If you’ve read it before, you’ll rediscover what made Don’t Make Me Think so essential to Web designers and developers around the world. If you’ve never read it, you’ll see why so many people have said it should be required reading for anyone working on Web sites.

    Now, internationally renowned performance expert Brendan Gregg has brought together proven methodologies, tools, and metrics for analyzing and tuning even the most complex environments. Systems Performance: Enterprise and the Cloud focuses on Linux(R) and Unix(R) performance, while illuminating performance issues that are relevant to all operating systems. You'll gain deep insight into how systems work and perform, and learn methodologies for analyzing and improving system and application performance. Gregg presents examples from bare-metal systems and virtualized cloud tenants running Linux-based Ubuntu(R), Fedora(R), CentOS, and the illumos-based Joyent(R) SmartOS(TM) and OmniTI OmniOS(R). He systematically covers modern systems performance, including the "traditional" analysis of CPUs, memory, disks, and networks, and new areas including cloud computing and dynamic tracing. This book also helps you identify and fix the "unknown unknowns" of complex performance: bottlenecks that emerge from elements and interactions you were not aware of. The text concludes with a detailed case study, showing how a real cloud customer issue was analyzed from start to finish. Coverage includes - Modern performance analysis and tuning: terminology, concepts, models, methods, and techniques - Dynamic tracing techniques and tools, including examples of DTrace, SystemTap, and perf - Kernel internals: uncovering what the OS is doing - Using system observability tools, interfaces, and frameworks - Understanding and monitoring application performance - Optimizing CPUs: processors, cores, hardware threads, caches, interconnects, and kernel scheduling - Memory optimization: virtual memory, paging, swapping, memory architectures, busses, address spaces, and allocators - File system I/O, including caching - Storage devices/controllers, disk I/O workloads, RAID, and kernel I/O - Network-related performance issues: protocols, sockets, interfaces, and physical connections - Performance implications of OS and hardware-based virtualization, and new issues encountered with cloud computing - Benchmarking: getting accurate results and avoiding common mistakes This guide is indispensable for anyone who operates enterprise or cloud environments: system, network, database, and web admins; developers; and other professionals. For students and others new to optimization, it also provides exercises reflecting Gregg's extensive instructional experience.

    But recently, the industry has turned to dynamic languages for increased productivity and speed to market.Groovy is one of a new breed of dynamic languages that run on the Java platform. You can use these new languages on the JVM and intermix them with your existing Java code. You can leverage your Java investments while benefiting from advanced features including true Closures, Meta Programming, the ability to create internal DSLs, and a higher level of abstraction.If you're an experienced Java developer, Programming Groovy will help you learn the necessary fundamentals of programming in Groovy. You'll see how to use Groovy to do advanced programming including using Meta Programming, Builders, Unit Testing with Mock objects, processing XML, working with Databases and creating your own Domain-Specific Languages (DSLs).

    Updated to cover Internet Explorer 7, Microsoft's vastly improved browser, this new edition includes content on positioning, lists and generated content, table layout, user interface, paged media, and more.Simply put, Cascading Style Sheets (CSS) is a way to separate a document's structure from its presentation. The benefits of this can be quite profound: CSS allows a much richer document appearance than HTML and also saves time -- you can create or change the appearance of an entire document in just one place; and its compact file size makes web pages load quickly.CSS: The Definitive Guide, 3rd Edition, provides you with a comprehensive guide to CSS implementation, along with a thorough review of all aspects of CSS 2.1. Updated to cover Internet Explorer 7, Microsoft's vastly improved browser, this new edition includes content on positioning, lists and generated content, table layout, user interface, paged media, and more. Author Eric Meyer tackles the subject with passion, exploring in detail each individual CSS property and how it interacts with other properties. You'll not only learn how to avoid common mistakes in interpretation, you also will benefit from the depth and breadth of his experience and his clear and honest style. This is the complete sourcebook on CSS.The 3rd edition contains:Updates to reflect changes in the latest draft version of CSS 2.1Browser notes updated to reflect changes between IE6 and IE7Advanced selectors supported in IE7 and other major browsers includedA new round of technical edits by a fresh set of editorsClarifications and corrected errata, including updated URLs ofreferenced online resources

    In this question-and-answer book on jQuery, you'll find a cookbook of ready-to-go solutions to help breathe life into your web page. Topics covered include: - Scrolling, Resizing and Animating Webpage elements - Backgrounds, Slideshows, and Crossfaders - Menus, Tabs, and Panels - Buttons, Fields, and Controls - Lists, Trees, and Tables - Frames, Windows, and Dialogs - Adding interactivity with Ajax - Using the jQuery User Interface Themeroller - Writing your own jQuery plug-ins All code used to create each solution is available for download and guaranteed to be simple, efficient and cross-browser compatible.