With an emphasis on improvements and new features in Spark 2.0, authors Bill Chambers and Matei Zaharia break down Spark topics into distinct sections, each with unique goals. You’ll explore the basic operations and common functions of Spark’s structured APIs, as well as Structured Streaming, a new high-level API for building end-to-end streaming applications. Developers and system administrators will learn the fundamentals of monitoring, tuning, and debugging Spark, and explore machine learning techniques and scenarios for employing MLlib, Spark’s scalable machine-learning library. Get a gentle overview of big data and Spark Learn about DataFrames, SQL, and Datasets—Spark’s core APIs—through worked examples Dive into Spark’s low-level APIs, RDDs, and execution of SQL and DataFrames Understand how Spark runs on a cluster Debug, monitor, and tune Spark clusters and applications Learn the power of Structured Streaming, Spark’s stream-processing engine Learn how you can apply MLlib to a variety of problems, including classification or recommendation

    Building on the successful top-down approach of previous editions, this fourth edition continues with an early emphasis on application-layer paradigms and application programming interfaces, encouraging a hands-on experience with protocols and networking concepts.

    The Web is awash in Ruby code that is now virtually impossible to change or extend. This text helps you solve that problem by using powerful real-world object-oriented design techniques, which it thoroughly explains using simple and practical Ruby examples. Sandi Metz has distilled a lifetime of conversations and presentations about object-oriented design into a set of Ruby-focused practices for crafting manageable, extensible, and pleasing code. She shows you how to build new applications that can survive success and repair existing applications that have become impossible to change. Each technique is illustrated with extended examples, all downloadable from the companion Web site, poodr.info. The first title to focus squarely on object-oriented Ruby application design, Practical Object-Oriented Design in Ruby will guide you to superior outcomes, whatever your previous Ruby experience. Novice Ruby programmers will find specific rules to live by; intermediate Ruby programmers will find valuable principles they can flexibly interpret and apply; and advanced Ruby programmers will find a common language they can use to lead development and guide their colleagues. This guide will help you Understand how object-oriented programming can help you craft Ruby code that is easier to maintain and upgrade Decide what belongs in a single Ruby class Avoid entangling objects that should be kept separate Define flexible interfaces among objects Reduce programming overhead costs with duck typing Successfully apply inheritance Build objects via composition Design cost-effective tests Solve common problems associated with poorly designed Ruby code

    Mastering Algorithms with C offers you a unique combination of theoretical background and working code. With robust solutions for everyday programming tasks, this book avoids the abstract style of most classic data structures and algorithms texts, but still provides all of the information you need to understand the purpose and use of common programming techniques.Implementations, as well as interesting, real-world examples of each data structure and algorithm, are included.Using both a programming style and a writing style that are exceptionally clean, Kyle Loudon shows you how to use such essential data structures as lists, stacks, queues, sets, trees, heaps, priority queues, and graphs. He explains how to use algorithms for sorting, searching, numerical analysis, data compression, data encryption, common graph problems, and computational geometry. And he describes the relative efficiency of all implementations. The compression and encryption chapters not only give you working code for reasonably efficient solutions, they offer explanations of concepts in an approachable manner for people who never have had the time or expertise to study them in depth.Anyone with a basic understanding of the C language can use this book. In order to provide maintainable and extendible code, an extra level of abstraction (such as pointers to functions) is used in examples where appropriate. Understanding that these techniques may be unfamiliar to some programmers, Loudon explains them clearly in the introductory chapters.Contents include:PointersRecursionAnalysis of algorithmsData structures (lists, stacks, queues, sets, hash tables, trees, heaps, priority queues, graphs)Sorting and searchingNumerical methodsData compressionData encryptionGraph algorithmsGeometric algorithms

    But facing the sea change that AI will bring can be paralyzing. How should companies set strategies, governments design policies, and people plan their lives for a world so different from what we know? In the face of such uncertainty, many analysts either cower in fear or predict an impossibly sunny future.But in Prediction Machines, three eminent economists recast the rise of AI as a drop in the cost of prediction. With this single, masterful stroke, they lift the curtain on the AI-is-magic hype and show how basic tools from economics provide clarity about the AI revolution and a basis for action by CEOs, managers, policy makers, investors, and entrepreneurs.When AI is framed as cheap prediction, its extraordinary potential becomes clear: Prediction is at the heart of making decisions under uncertainty. Our businesses and personal lives are riddled with such decisions. Prediction tools increase productivity--operating machines, handling documents, communicating with customers. Uncertainty constrains strategy. Better prediction creates opportunities for new business structures and strategies to compete. Penetrating, fun, and always insightful and practical, Prediction Machines follows its inescapable logic to explain how to navigate the changes on the horizon. The impact of AI will be profound, but the economic framework for understanding it is surprisingly simple.

    The authors have taken the mystery out of complex queries and explained principles and techniques with such clarity that a "Mere Mortal" will indeed be empowered to perform the superhuman. Do not walk past this book "--Graham Mandeno, Database Consultant""SQL Queries for Mere Mortals" provides a step-by-step, easy-to-read introduction to writing SQL queries. It includes hundreds of examples with detailed explanations. This book provides the tools you need to understand, modify, and create SQL queries"--Keith W. Hare, Convenor, ISO/IEC JTC1 SC32 WG3--the International SQL Standards Committee"I learned SQL primarily from the first edition of this book, and I am pleased to see a second edition of this book so that others can continue to benefit from its organized presentation of the language. Starting from how to design your tables so that SQL can be effective (a common problem for database beginners), and then continuing through the various aspects of SQL construction and capabilities, the reader can become a moderate expert upon completing the book and its samples. Learning how to convert a question in English into a meaningful SQL statement will greatly facilitate your mastery of the language. Numerous examples from real life will help you visualize how to use SQL to answer the questions about the data in your database. Just one of the "watch out for this trap" items will save you more than the cost of the book when you avoid that problem when writing your queries. I highly recommend this book if you want to tap the full potential of your database."--Kenneth D. Snell, Ph.D., Database Designer/Programmer"I don't think they do this in public schools any more, and it is a shame, but do you remember in the seventh and eighth grades when you learned to diagram a sentence? Those of you who do may no longer remember how you did it, but all of you do write better sentences because of it. John Viescas and Mike Hernandez must have remembered because they take everyday English queries and literally translate them into SQL. This is an important book for all database designers. It takes the complexity of mathematical Set Theory and of First Order Predicate Logic, as outlined in E. F. Codd's original treatise on relational database design, and makes it easy for anyone to understand. If you want an elementary- through intermediate-level course on SQL, this is the one book that is a requirement, no matter how many others you buy."--Arvin Meyer, MCP, MVP"Even in this day of wizards and code generators, successful database developers still require a sound knowledge of Structured Query Language (SQL, the standard language for communicating with most database systems). In this book, John and Mike do a marvelous job of making what's usually a dry and difficult subject come alive, presenting the material with humor in a logical manner, with plenty of relevant examples. I would say that this book should feature prominently in the collection on the bookshelf of all serious developers, except that I'm sure it'll get so much use that it won't spend much time on the shelf "-- Doug Steele, Microsoft Access Developer and author"Over the last several decades, SQL has evolved from a language known only to computer specialists to a widely used international standard of the computer industry. The number of new applications deployed each year using SQL now totals in the millions. If you are accessing corporate information from the Internet or from an internal network, you are probably using SQL. This new edition of "SQL Queries for Mere Mortals" helps new users learn the foundations of SQL queries, and is an essential reference guide for intermediate and advanced users.The accompanying CD contains five sample databases used for the example queries throughout the book in four different formats: Microsoft SQL Server 2000 and later, Microsoft Access 2000 and later, MySQL version 5.0 and later, and SQL scripts that can be used with most other implementations of the language.

    This book draws on material Michael created for his renowned Object Mentor seminars, techniques Michael has used in mentoring to help hundreds of developers, technical managers, and testers bring their legacy systems under control. The topics covered include: Understanding the mechanics of software change, adding features, fixing bugs, improving design, optimizing performance Getting legacy code into a test harness Writing tests that protect you against introducing new problems Techniques that can be used with any language or platform, with examples in Java, C++, C, and C# Accurately identifying where code changes need to be made Coping with legacy systems that aren't object-oriented Handling applications that don't seem to have any structureThis book also includes a catalog of twenty-four dependency-breaking techniques that help you work with program elements in isolation and make safer changes.

    This book attempts to capture the engineering wisdom and design philosophy of the UNIX, Linux, and Open Source software development community as it has evolved over the past three decades, and as it is applied today by the most experienced programmers. Eric Raymond offers the next generation of hackers the unique opportunity to learn the connection between UNIX philosophy and practice through careful case studies of the very best UNIX/Linux programs.

    

    Led by three renowned internals experts, this classic guide is fully updated for Windows 7 and Windows Server 2008 R2—and now presents its coverage in two volumes.As always, you get critical insider perspectives on how Windows operates. And through hands-on experiments, you’ll experience its internal behavior firsthand—knowledge you can apply to improve application design, debugging, system performance, and support.In Part 1, you will:Understand how core system and management mechanisms work—including the object manager, synchronization, Wow64, Hyper-V, and the registryExamine the data structures and activities behind processes, threads, and jobsGo inside the Windows security model to see how it manages access, auditing, and authorizationExplore the Windows networking stack from top to bottom—including APIs, BranchCache, protocol and NDIS drivers, and layered servicesDig into internals hands-on using the kernel debugger, performance monitor, and other tools

    Fueled by significant benefits and an impressive portfolio of real-world applications already in production, Rails is destined to continue making significant inroads in coming years.Each new Rails application showing up on the web adds yet more to the collective wisdom of the Rails development community. Yesterday's best practices yield to today's latest and greatest techniques, as the state of the art is continually refined in kitchens all across the Internet. Indeed, these are times of great progress.At the same time, it's easy to get left behind in the wake of progress. Advanced Rails Recipes keeps you on the cutting edge of Rails development and, more importantly, continues to turn this fast-paced framework to your advantage.Advanced Rails Recipes is filled with pragmatic recipes you'll use on every Rails project. And by taking the code in these recipes and slipping it into your application you'll not only deliver your application quicker, you'll do so with the confidence that it's done right.The book includes contributions from Aaron Batalion, Adam Keys, Adam Wiggins, Andre Lewis, Andrew Kappen, Benjamin Curtis, Ben Smith, Chris Bernard, Chris Haupt, Chris Wanstrath, Cody Fauser, Dan Benjamin, Dan Manges, Daniel Fischer, David Bock, David Chelimsky, David Heinemeier Hansson, Erik Hatcher, Ezra Zygmuntowicz, Geoffrey Grosenbach, Giles Bowkett, Greg Hansen, Gregg Pollack, Hemant Kumar, Hugh Bien, Jamie Orchard-Hays, Jamis Buck, Jared Haworth, Jarkko Laine, Jason LaPier, Jay Fields, John Dewey, Jonathan Dahl, Josep Blanquer, Josh Stephenson, Josh Susser, Kevin Clark, Luke Francl, Mark Bates, Marty Haught, Matthew Bass, Michael Slater, Mike Clark, Mike Hagedorn, Mike Mangino, Mike Naberezny, Mike Subelsky, Nathaniel Talbott, PJ Hyett, Patrick Reagan, Peter Marklund, Pierre-Alexandre Meyer, Rick Olson, Ryan Bates, Scott Barron, Tony Primerano, Val Aleksenko, and Warren Konkel.

    You'll find more than 200 tips and techniques for building a variety of objects and prototypes such as IoT solutions, environmental monitors, location and position-aware systems, and products that can respond to touch, sound, heat, and light.Updated for the Arduino 1.8 release, the recipes in this third edition include practical examples and guidance to help you begin, expand, and enhance your projects right away--whether you're an engineer, designer, artist, student, or hobbyist.Get up to speed on the Arduino board and essential software concepts quicklyLearn basic techniques for reading digital and analog signalsUse Arduino with a variety of popular input devices and sensorsDrive visual displays, generate sound, and control several types of motorsConnect Arduino to wired and wireless networksLearn techniques for handling time delays and time measurementApply advanced coding and memory-handling techniques

    When Ruby first burst onto the scene in the Western world, the Pragmatic Programmers were there with the definitive reference manual, Programming Ruby: The Pragmatic Programmer's Guide.Now in its second edition, author Dave Thomas has expanded the famous Pickaxe book with over 200 pages of new content, covering all the improved language features of Ruby 1.8 and standard library modules. The Pickaxe contains four major sections:An acclaimed tutorial on using Ruby.The definitive reference to the language.Complete documentation on all built-in classes, modules, and methodsComplete descriptions of all 98 standard libraries.If you enjoyed the First Edition, you'll appreciate the expanded content, including enhanced coverage of installation, packaging, documenting Ruby source code, threading and synchronization, and enhancing Ruby's capabilities using C-language extensions. Programming for the World Wide Web is easy in Ruby, with new chapters on XML/RPC, SOAP, distributed Ruby, templating systems, and other web services. There's even a new chapter on unit testing.This is the definitive reference manual for Ruby, including a description of all the standard library modules, a complete reference to all built-in classes and modules (including more than 250 significant changes since the First Edition). Coverage of other features has grown tremendously, including details on how to harness the sophisticated capabilities of irb, so you can dynamically examine and experiment with your running code. Ruby is a wonderfully powerful and useful language, and whenever I'm working with it this book is at my side --Martin Fowler, Chief Scientist, ThoughtWorks

    R is packed with powerful programming capabilities, but learning to use R in the real world can be overwhelming for even the most seasoned statisticians. This easy-to-follow guide explains how to use R for data processing and statistical analysis, and then, shows you how to present your data using compelling and informative graphics. You'll gain practical experience using R in a variety of settings and delve deeper into R's feature-rich toolset.Includes tips for the initial installation of RDemonstrates how to easily perform calculations on vectors, arrays, and lists of dataShows how to effectively visualize data using R's powerful graphics packagesGives pointers on how to find, install, and use add-on packages created by the R communityProvides tips on getting additional help from R mailing lists and websitesWhether you're just starting out with statistical analysis or are a procedural programming pro, "R For Dummies" is the book you need to get the most out of R.

    It's available on almost every OS--if you master the techniques in this book, you'll never need another text editor. Practical Vim shows you 120 vim recipes so you can quickly learn the editor's core functionality and tackle your trickiest editing and writing tasks. Vim, like its classic ancestor vi, is a serious tool for programmers, web developers, and sysadmins. No other text editor comes close to Vim for speed and efficiency; it runs on almost every system imaginable and supports most coding and markup languages. Learn how to edit text the "Vim way:" complete a series of repetitive changes with The Dot Formula, using one keystroke to strike the target, followed by one keystroke to execute the change. Automate complex tasks by recording your keystrokes as a macro. Run the same command on a selection of lines, or a set of files. Discover the "very magic" switch, which makes Vim's regular expression syntax more like Perl's. Build complex patterns by iterating on your search history. Search inside multiple files, then run Vim's substitute command on the result set for a project-wide search and replace. All without installing a single plugin! You'll learn how to navigate text documents as fast as the eye moves--with only a few keystrokes. Jump from a method call to its definition with a single command. Use Vim's jumplist, so that you can always follow the breadcrumb trail back to the file you were working on before. Discover a multilingual spell-checker that does what it's told.Practical Vim will show you new ways to work with Vim more efficiently, whether you're a beginner or an intermediate Vim user. All this, without having to touch the mouse.What You Need: Vim version 7