With this practical guide, you’ll learn how to use Python scripts and web APIs to gather and process data from thousands—or even millions—of web pages at once. Ideal for programmers, security professionals, and web administrators familiar with Python, this book not only teaches basic web scraping mechanics, but also delves into more advanced topics, such as analyzing raw data or using scrapers for frontend website testing. Code samples are available to help you understand the concepts in practice. Learn how to parse complicated HTML pages Traverse multiple pages and sites Get a general overview of APIs and how they work Learn several methods for storing the data you scrape Download, read, and extract data from documents Use tools and techniques to clean badly formatted data Read and write natural languages Crawl through forms and logins Understand how to scrape JavaScript Learn image processing and text recognition

    You can think of the Python programming language as your tool to solve data problems that are beyond the capability of a spreadsheet.Python is an easy to use and easy to learn programming language that is freely available on Macintosh, Windows, or Linux computers. So once you learn Python you can use it for the rest of your career without needing to purchase any software.This book uses the Python 3 language. The earlier Python 2 version of this book is titled "Python for Informatics: Exploring Information".

    But if one per year is good, how about Seven Languages in Seven Weeks? In this book you'll get a hands-on tour of Clojure, Haskell, Io, Prolog, Scala, Erlang, and Ruby. Whether or not your favorite language is on that list, you'll broaden your perspective of programming by examining these languages side-by-side. You'll learn something new from each, and best of all, you'll learn how to learn a language quickly. Ruby, Io, Prolog, Scala, Erlang, Clojure, Haskell. With Seven Languages in Seven Weeks, by Bruce A. Tate, you'll go beyond the syntax-and beyond the 20-minute tutorial you'll find someplace online. This book has an audacious goal: to present a meaningful exploration of seven languages within a single book. Rather than serve as a complete reference or installation guide, Seven Languages hits what's essential and unique about each language. Moreover, this approach will help teach you how to grok new languages. For each language, you'll solve a nontrivial problem, using techniques that show off the language's most important features. As the book proceeds, you'll discover the strengths and weaknesses of the languages, while dissecting the process of learning languages quickly--for example, finding the typing and programming models, decision structures, and how you interact with them. Among this group of seven, you'll explore the most critical programming models of our time. Learn the dynamic typing that makes Ruby, Python, and Perl so flexible and compelling. Understand the underlying prototype system that's at the heart of JavaScript. See how pattern matching in Prolog shaped the development of Scala and Erlang. Discover how pure functional programming in Haskell is different from the Lisp family of languages, including Clojure. Explore the concurrency techniques that are quickly becoming the backbone of a new generation of Internet applications. Find out how to use Erlang's let-it-crash philosophy for building fault-tolerant systems. Understand the actor model that drives concurrency design in Io and Scala. Learn how Clojure uses versioning to solve some of the most difficult concurrency problems. It's all here, all in one place. Use the concepts from one language to find creative solutions in another-or discover a language that may become one of your favorites.

    Writing code that handles unexpected errors and still works is really hard. Most of us learn by trial and error. This short book removes the uncertainty. With over 100 pages of content and dozens of working examples, you’ll learn everything from the mechanics of how exceptions work to how to design a robust failure management architecture for your app or library. Whether you are a Ruby novice or a seasoned veteran, Exceptional Ruby will help you write cleaner, more resilient Ruby code.

    With this hands-on book, you’ll learn Flask from the ground up by developing a complete social blogging application step-by-step. Author Miguel Grinberg walks you through the framework’s core functionality, and shows you how to extend applications with advanced web techniques such as database migration and web service communication.Rather than impose development guidelines as other frameworks do, Flask leaves the business of extensions up to you. If you have Python experience, this book shows you how to take advantage of that creative freedom.- Learn Flask’s basic application structure and write an example app- Work with must-have components—templates, databases, web forms, and email support- Use packages and modules to structure a large application that scales- Implement user authentication, roles, and profiles- Build a blogging feature by reusing templates, paginating item lists, and working with rich text- Use a Flask-based RESTful API to expose app functionality to smartphones, tablets, and other third-party clients- Learn how to run unit tests and enhance application performance- Explore options for deploying your web app to a production server

    This update to the bestselling first edition dives in to cover the exciting new features of the latest release of the Android mobile platform.Providing in-depth coverage of how to build mobile applications using the next major release of the Android SDK, this invaluable resource takes a hands-on approach to discussing Android with a series of projects, each of which introduces a new feature and highlights techniques and best practices to get the most out of Android.The Android SDK is a powerful, flexible, open source platform for mobile devices Shares helpful techniques and best practices to maximize the capabilities of Android Explains the possibilities of Android through the use of a series of detailed projects Demonstrates how to create real-world mobile applications for Android phones Includes coverage of the latest version of Android Providing concise and compelling examples, Professional Android Application Development is an updated guide aimed at helping you create mobile applications for mobile devices running the latest version of Android.

    You'll learn the SPA design approach, and then start exploring new techniques like structured JavaScript and responsive design. And you'll learn how to capitalize on trends like server-side JavaScript and NoSQL data stores, as well as new frameworks that make JavaScript more manageable and testable as a first-class language.About this BookIf your website is a jumpy collection of linked pages, you are behind. Single page web applications are your next step: pushing UI rendering and business logic to the browser and communicating with the server only to synchronize data, they provide a smooth user experience, much like a native application. But, SPAs can be hard to develop, manage, and test.Single Page Web Applications shows how your team can easily design, test, maintain, and extend sophisticated SPAs using JavaScript end-to-end, without getting locked into a framework. Along the way, you'll develop advanced HTML5, CSS3, and JavaScript skills, and use JavaScript as the language of the web server and the database.This book assumes basic knowledge of web development. No experience with SPAs is required.Purchase of the print book includes a free eBook in PDF, Kindle, and ePub formats from Manning Publications.What's InsideDesign, build, and test a full-stack SPA Best-in-class tools like jQuery, TaffyDB, Node.js, and MongoDB Real-time web with web sockets and Socket.IO Touch controls for tablets and smartphones Common SPA design mistakesAbout the AuthorsThe authors are architects and engineering managers. Michael Mikowski has worked on many commercial SPAs and a platform that processes over 100 billion requests per year. Josh Powell has built some of the most heavily trafficked sites on the web.Table of ContentsPART 1: INTRODUCING SPAS Our first single page application Reintroducing JavaScript PART 2: SPA CLIENT Develop the Shell Add feature modules Build the Model Finish the Model and Data modules PART 3: THE SPA SERVER The web server The server database Readying our SPA for production

    As a result, functional code is easier to test and reuse, simpler to parallelize, and less prone to bugs. Scala is an emerging JVM language that offers strong support for FP. Its familiar syntax and transparent interoperability with existing Java libraries make Scala a great place to start learning FP.Functional Programming in Scala is a serious tutorial for programmers looking to learn FP and apply it to the everyday business of coding. The book guides readers from basic techniques to advanced topics in a logical, concise, and clear progression. In it, they'll find concrete examples and exercises that open up the world of functional programming.Purchase of the print book comes with an offer of a free PDF, ePub, and Kindle eBook from Manning. Also available is all code from the book.

    Written in a frank but engaging style, Debuggingprovides simple, foolproof principles guaranteed to help find any bug quickly. This book makes those shelves of application-specific debugging books (on C++, Perl, Java, etc.) obsolete. It changes the way readers think about debugging, making those pesky problems suddenly much easier to find and fix. Illustrating the rules with real-life bug-detection war stories, the book shows readers how to: * Understand the system: how perceiving the ""roadmap"" can hasten your journey * Quit thinking and look: when hands-on investigation can’t be avoided * Isolate critical factors: why changing one element at a time can be an essential tool * Keep an audit trail: how keeping a record of the debugging process can win the day

    Legendary testing expert James Whittaker, until recently a Google testing leader, and two top Google experts reveal exactly how Google tests software, offering brand-new best practices you can use even if you're not quite Google's size...yet! Breakthrough Techniques You Can Actually Use Discover 100% practical, amazingly scalable techniques for analyzing risk and planning tests...thinking like real users...implementing exploratory, black box, white box, and acceptance testing...getting usable feedback...tracking issues...choosing and creating tools...testing "Docs & Mocks," interfaces, classes, modules, libraries, binaries, services, and infrastructure...reviewing code and refactoring...using test hooks, presubmit scripts, queues, continuous builds, and more. With these techniques, you can transform testing from a bottleneck into an accelerator-and make your whole organization more productive!

    Now, for the first time, leading agile methodologist Scott Ambler and renowned consultantPramodkumar Sadalage introduce powerful refactoring techniquesspecifically designed for database systems. Ambler and Sadalagedemonstrate how small changes to table structures, data, storedprocedures, and triggers can significantly enhance virtually anydatabase design - without changing semantic

    By reconstructing authentic, complex AI programs using state-of-the-art Common Lisp, the book teaches students and professionals how to build and debug robust practical programs, while demonstrating superior programming style and important AI concepts. The author strongly emphasizes the practical performance issues involved in writing real working programs of significant size. Chapters on troubleshooting and efficiency are included, along with a discussion of the fundamentals of object-oriented programming and a description of the main CLOS functions. This volume is an excellent text for a course on AI programming, a useful supplement for general AI courses and an indispensable reference for the professional programmer.

     Master each chapter with free interactive exercises online. Live simulation lets you see your practice code run in your browser. 2,000 lines of color-keyed sample code break it all down into easy-to-learn chunks. Extra help through the rough spots so you're less likely to get stuck. Tested on non-coders—including the author's technophobe wife. Become fluent in all the JavaScript fundamentals, in half the time. Display alert messages to the user Gather information through prompts Manipulate variables Build statements Do math Use operators Concatenate text Run routines based on conditions Compare values Work with arrays Run automated routines Display custom elements on the webpage Generate random numbers Manipulate decimals Round numbers Create loops Use functions Find the current date and time Measure time intervals Create a timer Respond to the user's actions Swap images Control colors on the webpage Change any element on the webpage Improvise new HTML markup on the fly Use the webpage DOM structure Insert comments Situate scripts effectively Create and change objects Automate object creation Control the browser's actions Fill the browser window with custom content Check forms for invalid entries Deal with errors Make a more compelling website Increase user-friendliness Keep your user engaged

    Generics and the greatly expanded collection libraries have tremendously increased the power of Java 5 and Java 6. But they have also confused many developers who haven't known how to take advantage of these new features.Java Generics and Collections covers everything from the most basic uses of generics to the strangest corner cases. It teaches you everything you need to know about the collections libraries, so you'll always know which collection is appropriate for any given task, and how to use it.Topics covered include:• Fundamentals of generics: type parameters and generic methods• Other new features: boxing and unboxing, foreach loops, varargs• Subtyping and wildcards• Evolution not revolution: generic libraries with legacy clients and generic clients with legacy libraries• Generics and reflection• Design patterns for generics• Sets, Queues, Lists, Maps, and their implementations• Concurrent programming and thread safety with collections• Performance implications of different collectionsGenerics and the new collection libraries they inspired take Java to a new level. If you want to take your software development practice to a new level, this book is essential reading.Philip Wadler is Professor of Theoretical Computer Science at the University of Edinburgh, where his research focuses on the design of programming languages. He is a co-designer of GJ, work that became the basis for generics in Sun's Java 5.0.Maurice Naftalin is Technical Director at Morningside Light Ltd., a software consultancy in the United Kingdom. He has most recently served as an architect and mentor at NSB Retail Systems plc, and as the leader of the client development team of a major UK government social service system."A brilliant exposition of generics. By far the best book on the topic, it provides a crystal clear tutorial that starts with the basics and ends leaving the reader with a deep understanding of both the use and design of generics." Gilad Bracha, Java Generics Lead, Sun Microsystems

    With Java 5.0, you'll not only find substantial changes in the platform, but to the language itself-something that developers of Java took five years to complete. The main goal of Java 5.0 is to make it easier for you to develop safe, powerful code, but none of these improvements makes Java any easier to learn, even if you've programmed with Java for years. And that means our bestselling hands-on tutorial takes on even greater significance."Learning Java" is the most widely sought introduction to the programming language that's changed the way we think about computing. Our updated third edition takes an objective, no-nonsense approach to the new features in Java 5.0, some of which are drastically different from the way things were done in any previous versions. The most essential change is the addition of "generics," a feature that allows developers to write, test, and deploy code once, and then reuse the code again and again for different data types. The beauty of generics is that more problems will be caught during development, and "Learning Java" will show you exactly how it's done.Java 5.0 also adds more than 1,000 new classes to the Java library. That means 1,000 new things you can do without having to program it in yourself. That's a huge change. With our book's practical examples, you'll come up to speed quickly on this and other new features such as loops and threads. The new edition also includes an introduction to Eclipse, the open source IDE that is growing in popularity. "Learning Java," 3rd Edition addresses all of the important uses of Java, such as web applications, servlets, and XML that are increasingly driving enterprise applications.