But if you haven't seen the performance improvements you expected, or still don't feel confident enough to use Spark in production, this practical book is for you. Authors Holden Karau and Rachel Warren demonstrate performance optimizations to help your Spark queries run faster and handle larger data sizes, while using fewer resources.Ideal for software engineers, data engineers, developers, and system administrators working with large-scale data applications, this book describes techniques that can reduce data infrastructure costs and developer hours. Not only will you gain a more comprehensive understanding of Spark, you'll also learn how to make it sing.With this book, you'll explore:How Spark SQL's new interfaces improve performance over SQL's RDD data structureThe choice between data joins in Core Spark and Spark SQLTechniques for getting the most out of standard RDD transformationsHow to work around performance issues in Spark's key/value pair paradigmWriting high-performance Spark code without Scala or the JVMHow to test for functionality and performance when applying suggested improvementsUsing Spark MLlib and Spark ML machine learning librariesSpark's Streaming components and external community packages

     At any given moment, somewhere in the world someone struggles with the same software design problems you have. You know you don't want to reinvent the wheel (or worse, a flat tire), so you look to Design Patterns--the lessons learned by those who've faced the same problems. With Design Patterns, you get to take advantage of the best practices and experience of others, so that you can spend your time on...something else. Something more challenging. Something more complex. Something more fun. You want to learn about the patterns that matter--why to use them, when to use them, how to use them (and when NOT to use them). But you don't just want to see how patterns look in a book, you want to know how they look "in the wild". In their native environment. In other words, in real world applications. You also want to learn how patterns are used in the Java API, and how to exploit Java's built-in pattern support in your own code. You want to learn the real OO design principles and why everything your boss told you about inheritance might be wrong (and what to do instead). You want to learn how those principles will help the next time you're up a creek without a design pattern. Most importantly, you want to learn the "secret language" of Design Patterns so that you can hold your own with your co-worker (and impress cocktail party guests) when he casually mentions his stunningly clever use of Command, Facade, Proxy, and Factory in between sips of a martini. You'll easily counter with your deep understanding of why Singleton isn't as simple as it sounds, how the Factory is so often misunderstood, or on the real relationship between Decorator, Facade and Adapter. With Head First Design Patterns, you'll avoid the embarrassment of thinking Decorator is something from the "Trading Spaces" show. Best of all, in a way that won't put you to sleep! We think your time is too important (and too short) to spend it struggling with academic texts. If you've read a Head First book, you know what to expect--a visually rich format designed for the way your brain works. Using the latest research in neurobiology, cognitive science, and learning theory, Head First Design Patterns will load patterns into your brain in a way that sticks. In a way that lets you put them to work immediately. In a way that makes you better at solving software design problems, and better at speaking the language of patterns with others on your team.

    With the 97 short and extremely useful tips for programmers in this book, you'll expand your skills by adopting new approaches to old problems, learning appropriate best practices, and honing your craft through sound advice.With contributions from some of the most experienced and respected practitioners in the industry--including Michael Feathers, Pete Goodliffe, Diomidis Spinellis, Cay Horstmann, Verity Stob, and many more--this book contains practical knowledge and principles that you can apply to all kinds of projects.A few of the 97 things you should know:"Code in the Language of the Domain" by Dan North"Write Tests for People" by Gerard Meszaros"Convenience Is Not an -ility" by Gregor Hohpe"Know Your IDE" by Heinz Kabutz"A Message to the Future" by Linda Rising"The Boy Scout Rule" by Robert C. Martin (Uncle Bob)"Beware the Share" by Udi Dahan

    Others have the knack of unerringly zeroing in on the root cause of a bug. Are they geniuses? Just lucky? No, they've learned the secrets of professional debugging. This book will equip you with the tools, techniques and approaches-proven in the crucible of professional software development-to ensure that you can tackle any bug with confidence. You'll learn how to handle every stage of the bug life-cycle, from constructing software that makes debugging easy, through detection, reproduction, diagnosis and rolling out your eventual fix. If you develop software, sooner or later you're going to discover that it doesn't always behave as you intended. Working out why it's misbehaving can be hard. Sometimes very hard. Debug It! is here to help! All bugs are different: there is no silver bullet. You've got to rely upon your intellect, intuition, detective skills and yes, even a little luck. But that doesn't mean that you're completely on your own-there is much you can learn from those who have gone before. This book distills decades of hard-won experience gained in the trenches of professional software development, giving you a head-start and arming you with the tools you need to get to the bottom of the problem, whatever you're faced with. Whether you're writing Java or assembly language, targeting servers or embedded micro-controllers, using agile or traditional approaches, the same basic bug-fixing principles apply. From constructing software that is easy to debug (and incidentally less likely to contain bugs in the first place), through handling bug reports to rolling out your ultimate fix, we'll cover the entire life-cycle of a bug. You'll learn about the empirical approach, which leverages your software's unique ability to show you what's really happening, the importance of finding a reliable and convenient means of reproducing a bug, and common pitfalls so you can avoid them. You'll see how to use commonly available tools to automatically detect problems before they're reported by customers and how to construct "transparent software" that provides access to critical information and internal state.

    The revised edition is updated with a greater focus on digital recording techniques, the most powerful tools available to the home recordist. There are chapters devoted to instrument recording, humanizing drum patterns, mixing with plug-ins and virtual consoles, and a new section on using digital audio skills. And since, many true "Guerrillas" still record to analog tape, we have retained the best of that world. This edition features many more graphics than in the original edition, further enforcing Guerrilla Home Recording's reputation as the most readable, user-frienly recording title on the market.

    With 97 short and extremely useful tips for engineering managers, you'll discover new approaches to old problems, pick up road-tested best practices, and hone your management skills through sound advice.Managing people is hard, and the industry as a whole is bad at it. Many managers lack the experience, training, tools, texts, and frameworks to do it well. From mentoring interns to working in senior management, this book will take you through the stages of management and provide actionable advice on how to approach the obstacles you'll encounter as a technical manager.A few of the 97 things you should know:"Three Ways to Be the Manager Your Report Needs" by Duretti Hirpa"The First Two Questions to Ask When Your Team Is Struggling" by Cate Huston"Fire Them!" by Mike Fisher"The 5 Whys of Organizational Design" by Kellan Elliott-McCrea"Career Conversations" by Raquel V�lez"Using 6-Page Documents to Close Decisions" by Ian Nowland"Ground Rules in Meetings" by Lara Hogan

    Emphasis is placed on the basic principles and priorities of the software process. Sections are organized in a natural way for managers and professionals to implement improvement activities.

    Significant numbers of poorly designed programs have been created by less-experienced developers, resulting in applications that are inefficient and hard to maintain and extend. Increasingly, software system professionals are discovering just how difficult it is to work with these inherited, non-optimal applications. For several years, expert-level object programmers have employed a growing collection of techniques to improve the structural integrity and performance of such existing software programs. Referred to as refactoring, these practices have remained in the domain of experts because no attempt has been made to transcribe the lore into a form that all developers could use... until now. In Refactoring: Improving the Design of Existing Software, renowned object technology mentor Martin Fowler breaks new ground, demystifying these master practices and demonstrating how software practitioners can realize the significant benefits of this new process.

    Now this classic book has been fully updated and revised with leading-edge practices--and hundreds of new code samples--illustrating the art and science of software construction. Capturing the body of knowledge available from research, academia, and everyday commercial practice, McConnell synthesizes the most effective techniques and must-know principles into clear, pragmatic guidance. No matter what your experience level, development environment, or project size, this book will inform and stimulate your thinking--and help you build the highest quality code. Discover the timeless techniques and strategies that help you: Design for minimum complexity and maximum creativity Reap the benefits of collaborative development Apply defensive programming techniques to reduce and flush out errors Exploit opportunities to refactor--or evolve--code, and do it safely Use construction practices that are right-weight for your project Debug problems quickly and effectively Resolve critical construction issues early and correctly Build quality into the beginning, middle, and end of your project

    Everyone has heard of computer programmers but few people realize there are nearly as many people behind the scenes with job titles such as Software Tester, Software Quality Assurance Engineer, Software Test Engineer, and Software Test Technician. Microsoft alone hires hundreds of people for these positions each year. There are also many companies whose sole purpose is providing software test consulting and software testing services. The first edition of Software Testing was published in November 2000. Although the processes and techniques used in testing computer software are timeless, this title will be brought up-to-date by adding a chapter that specifically deals with testing software for security bugs and revisiting the rest of the book to update examples and references.

    Did you design your system to survivef a sudden rush of visitors from Digg or Slashdot? Or an influx of real world customers from 100 different countries? Are you ready for a world filled with flakey networks, tangled databases, and impatient users?If you're a developer and don't want to be on call for 3AM for the rest of your life, this book will help.In Release It!, Michael T. Nygard shows you how to design and architect your application for the harsh realities it will face. You'll learn how to design your application for maximum uptime, performance, and return on investment.Mike explains that many problems with systems today start with the design.

    Advocates of NoSQL databases claim they can be used to build systems that are more performant, scale better, and are easier to program." ""NoSQL Distilled" is a concise but thorough introduction to this rapidly emerging technology. Pramod J. Sadalage and Martin Fowler explain how NoSQL databases work and the ways that they may be a superior alternative to a traditional RDBMS. The authors provide a fast-paced guide to the concepts you need to know in order to evaluate whether NoSQL databases are right for your needs and, if so, which technologies you should explore further. The first part of the book concentrates on core concepts, including schemaless data models, aggregates, new distribution models, the CAP theorem, and map-reduce. In the second part, the authors explore architectural and design issues associated with implementing NoSQL. They also present realistic use cases that demonstrate NoSQL databases at work and feature representative examples using Riak, MongoDB, Cassandra, and Neo4j. In addition, by drawing on Pramod Sadalage's pioneering work, "NoSQL Distilled" shows how to implement evolutionary design with schema migration: an essential technique for applying NoSQL databases. The book concludes by describing how NoSQL is ushering in a new age of Polyglot Persistence, where multiple data-storage worlds coexist, and architects can choose the technology best optimized for each type of data access.

    In today’s economy, we have far too much dismay along with our amazement, and technology bears some of the blame. In this combination of memoir, business strategy guide, and call to action, Tim O'Reilly, Silicon Valley’s leading intellectual and the founder of O’Reilly Media, explores the upside and the potential downsides of today's WTF? technologies. What is the future when an increasing number of jobs can be performed by intelligent machines instead of people, or done only by people in partnership with those machines? What happens to our consumer based societies—to workers and to the companies that depend on their purchasing power? Is income inequality and unemployment an inevitable consequence of technological advancement, or are there paths to a better future? What will happen to business when technology-enabled networks and marketplaces are better at deploying talent than traditional companies? How should companies organize themselves to take advantage of these new tools? What’s the future of education when on-demand learning outperforms traditional institutions? How can individuals continue to adapt and retrain? Will the fundamental social safety nets of the developed world survive the transition, and if not, what will replace them? O'Reilly is "the man who can really can make a whole industry happen," according to Eric Schmidt, Executive Chairman of Alphabet (Google.) His genius over the past four decades has been to identify and to help shape our response to emerging technologies with world shaking potential—the World Wide Web, Open Source Software, Web 2.0, Open Government data, the Maker Movement, Big Data, and now AI. O’Reilly shares the techniques he's used at O’Reilly Media  to make sense of and predict past innovation waves and applies those same techniques to provide a framework for thinking about how today’s world-spanning platforms and networks, on-demand services, and artificial intelligence are changing the nature of business, education, government, financial markets, and the economy as a whole. He provides tools for understanding how all the parts of modern digital businesses work together to create marketplace advantage and customer value, and why ultimately, they cannot succeed unless their ecosystem succeeds along with them.The core of the book's call to action is an exhortation to businesses to DO MORE with technology rather than just using it to cut costs and enrich their shareholders. Robots are going to take our jobs, they say. O'Reilly replies, “Only if that’s what we ask them to do! Technology is the solution to human problems, and we won’t run out of work till we run out of problems." Entrepreneurs need to set their sights on how they can use big data, sensors, and AI to create amazing human experiences and the economy of the future, making us all richer in the same way the tools of the first industrial revolution did. Yes, technology can eliminate labor and make things cheaper, but at its best, we use it to do things that were previously unimaginable! What is our poverty of imagination? What are the entrepreneurial leaps that will allow us to use the technology of today to build a better future, not just a more efficient one? Whether technology brings the WTF? of wonder or the WTF? of dismay isn't inevitable. It's up to us!

    The author incudes OOD, UML, Design Patterns, Agile and XP methods with a detailed description of a complete software design for reusable programs in C++ and Java. Using a practical, problem-solving approach, it shows how to develop an object-oriented application -- from the early stages of analysis, through the low-level design and into the implementation. Walks readers through the designer's thoughts -- showing the errors, blind alleys, and creative insights that occur throughout the software design process. KEY TOPICS: Covers: Statics and Dynamics; Principles of Class Design; Complexity Management; Principles of Package Design; Analysis and Design; Patterns and Paradigm Crossings. Explains the principles of OOD, one by one, and then demonstrates them with numerous examples, completely worked-through designs, and case studies. Covers traps, pitfalls, and work arounds in the application of C++ and OOD and then shows how Agile methods can be used. Discusses the methods for designing and developing big software in detail. Features a three-chapter, in-depth, single case study of a building security system. MARKET: For Software Engineers, Programmers, and Analysts who want to understand how to design object oriented software with state of the art methods.

    As the words "at work" suggest, Peter Seibel focuses on how his interviewees tackle the day–to–day work of programming, while revealing much more, like how they became great programmers, how they recognize programming talent in others, and what kinds of problems they find most interesting. Hundreds of people have suggested names of programmers to interview on the Coders at Work web site: http://www.codersatwork.com. The complete list was 284 names. Having digested everyone’s feedback, we selected 16 folks who’ve been kind enough to agree to be interviewed:- Frances Allen: Pioneer in optimizing compilers, first woman to win the Turing Award (2006) and first female IBM fellow- Joe Armstrong: Inventor of Erlang- Joshua Bloch: Author of the Java collections framework, now at Google- Bernie Cosell: One of the main software guys behind the original ARPANET IMPs and a master debugger- Douglas Crockford: JSON founder, JavaScript architect at Yahoo!- L. Peter Deutsch: Author of Ghostscript, implementer of Smalltalk-80 at Xerox PARC and Lisp 1.5 on PDP-1- Brendan Eich: Inventor of JavaScript, CTO of the Mozilla Corporation - Brad Fitzpatrick: Writer of LiveJournal, OpenID, memcached, and Perlbal - Dan Ingalls: Smalltalk implementor and designer- Simon Peyton Jones: Coinventor of Haskell and lead designer of Glasgow Haskell Compiler- Donald Knuth: Author of The Art of Computer Programming and creator of TeX- Peter Norvig: Director of Research at Google and author of the standard text on AI- Guy Steele: Coinventor of Scheme and part of the Common Lisp Gang of Five, currently working on Fortress- Ken Thompson: Inventor of UNIX- Jamie Zawinski: Author of XEmacs and early Netscape/Mozilla hackerWhat you’ll learn:How the best programmers in the world do their jobWho is this book for?Programmers interested in the point of view of leaders in the field. Programmers looking for approaches that work for some of these outstanding programmers.