Ideal for processing large datasets, the Apache Hadoop framework is an open source implementation of the MapReduce algorithm on which Google built its empire. This comprehensive resource demonstrates how to use Hadoop to build reliable, scalable, distributed systems: programmers will find details for analyzing large datasets, and administrators will learn how to set up and run Hadoop clusters. Complete with case studies that illustrate how Hadoop solves specific problems, this book helps you:Use the Hadoop Distributed File System (HDFS) for storing large datasets, and run distributed computations over those datasets using MapReduce Become familiar with Hadoop's data and I/O building blocks for compression, data integrity, serialization, and persistence Discover common pitfalls and advanced features for writing real-world MapReduce programs Design, build, and administer a dedicated Hadoop cluster, or run Hadoop in the cloud Use Pig, a high-level query language for large-scale data processing Take advantage of HBase, Hadoop's database for structured and semi-structured data Learn ZooKeeper, a toolkit of coordination primitives for building distributed systems If you have lots of data -- whether it's gigabytes or petabytes -- Hadoop is the perfect solution. Hadoop: The Definitive Guide is the most thorough book available on the subject. "Now you have the opportunity to learn about Hadoop from a master-not only of the technology, but also of common sense and plain talk." -- Doug Cutting, Hadoop Founder, Yahoo!

    You’ll learn how each of these platform-agnostic patterns work, when they might be useful in the cloud, and what impact they’ll have on your application architecture. You’ll also see an example of each pattern applied to an application built with Windows Azure.The patterns are organized into four major topics, such as scalability and handling failure, and primer chapters provide background on each topic. With the information in this book, you’ll be able to make informed decisions for designing effective cloud-native applications that maximize the value of cloud services, while also paying attention to user experience and operational efficiency.Learn about architectural patterns for:Scalability. Discover the advantages of horizontal scaling. Patterns covered include Horizontally Scaling Compute, Queue-Centric Workflow, and Auto-Scaling.Big data. Learn how to handle large amounts of data across a distributed system. Eventual consistency is explained, along with the MapReduce and Database Sharding patterns.Handling failure. Understand how multitenant cloud services and commodity hardware influence your applications. Patterns covered include Busy Signal and Node Failure.Distributed users. Learn how to overcome delays due to network latency when building applications for a geographically distributed user base. Patterns covered include Colocation, Valet Key, CDN, and Multi-Site Deployment.

    In 1999, "Refactoring" revolutionized design by introducing an effective process for improving code. With the highly anticipated " Refactoring to Patterns ," Joshua Kerievsky has changed our approach to design by forever uniting patterns with the evolutionary process of refactoring.This book introduces the theory and practice of pattern-directed refactorings: sequences of low-level refactorings that allow designers to safely move designs to, towards, or away from pattern implementations. Using code from real-world projects, Kerievsky documents the thinking and steps underlying over two dozen pattern-based design transformations. Along the way he offers insights into pattern differences and how to implement patterns in the simplest possible ways.Coverage includes: A catalog of twenty-seven pattern-directed refactorings, featuring real-world code examples Descriptions of twelve design smells that indicate the need for this book s refactorings General information and new insights about patterns and refactoringDetailed implementation mechanics: how low-level refactorings are combined to implement high-level patterns Multiple ways to implement the same pattern and when to use each Practical ways to get started even if you have little experience with patterns or refactoring"Refactoring to Patterns" reflects three years of refinement and the insights of more than sixty software engineering thought leaders in the global patterns, refactoring, and agile development communities. Whether you re focused on legacy or greenfield development, this book will make you a better software designer by helping you learn how to make important design changes safely and effectively. "

    As the words "at work" suggest, Peter Seibel focuses on how his interviewees tackle the day–to–day work of programming, while revealing much more, like how they became great programmers, how they recognize programming talent in others, and what kinds of problems they find most interesting. Hundreds of people have suggested names of programmers to interview on the Coders at Work web site: http://www.codersatwork.com. The complete list was 284 names. Having digested everyone’s feedback, we selected 16 folks who’ve been kind enough to agree to be interviewed:- Frances Allen: Pioneer in optimizing compilers, first woman to win the Turing Award (2006) and first female IBM fellow- Joe Armstrong: Inventor of Erlang- Joshua Bloch: Author of the Java collections framework, now at Google- Bernie Cosell: One of the main software guys behind the original ARPANET IMPs and a master debugger- Douglas Crockford: JSON founder, JavaScript architect at Yahoo!- L. Peter Deutsch: Author of Ghostscript, implementer of Smalltalk-80 at Xerox PARC and Lisp 1.5 on PDP-1- Brendan Eich: Inventor of JavaScript, CTO of the Mozilla Corporation - Brad Fitzpatrick: Writer of LiveJournal, OpenID, memcached, and Perlbal - Dan Ingalls: Smalltalk implementor and designer- Simon Peyton Jones: Coinventor of Haskell and lead designer of Glasgow Haskell Compiler- Donald Knuth: Author of The Art of Computer Programming and creator of TeX- Peter Norvig: Director of Research at Google and author of the standard text on AI- Guy Steele: Coinventor of Scheme and part of the Common Lisp Gang of Five, currently working on Fortress- Ken Thompson: Inventor of UNIX- Jamie Zawinski: Author of XEmacs and early Netscape/Mozilla hackerWhat you’ll learn:How the best programmers in the world do their jobWho is this book for?Programmers interested in the point of view of leaders in the field. Programmers looking for approaches that work for some of these outstanding programmers.

    Rust's modern, flexible types ensure your program is free of null pointer dereferences, double frees, dangling pointers, and similar bugs, all at compile time, without runtime overhead. In multi-threaded code, Rust catches data races at compile time, making concurrency much easier to use.Written by two experienced systems programmers, this book explains how Rust manages to bridge the gap between performance and safety, and how you can take advantage of it. Topics include:How Rust represents values in memory (with diagrams)Complete explanations of ownership, moves, borrows, and lifetimesCargo, rustdoc, unit tests, and how to publish your code on crates.io, Rust's public package repositoryHigh-level features like generic code, closures, collections, and iterators that make Rust productive and flexibleConcurrency in Rust: threads, mutexes, channels, and atomics, all much safer to use than in C or C++Unsafe code, and how to preserve the integrity of ordinary code that uses itExtended examples illustrating how pieces of the language fit together

    Along with a clear introduction to asynchronous programming, you get an in-depth look at how the async feature works and why you might want to use it in your application.Written for experienced C# programmers—yet approachable for beginners—this book is packed with code examples that you can extend for your own projects.Write your own asynchronous code, and learn how async saves you from this messy choreDiscover new performance possibilities in ASP.NET web server codeExplore how async and WinRT work together in Windows 8 applicationsLearn the importance of the await keyword in async methodsUnderstand which .NET thread is running your code—and at what points in the programUse the Task-based Asynchronous Pattern (TAP) to write asynchronous APIs in .NETTake advantage of parallel computing in modern machinesMeasure async code performance by comparing it with alternatives

    Ideal as a quick reference, Regular Expression Pocket Reference covers the regular expression APIs for Perl 5.8, Ruby (including some upcoming 1.9 features), Java, PHP, .NET and C#, Python, vi, JavaScript, and the PCRE regular expression libraries. This concise and easy-to-use reference puts a very powerful tool for manipulating text and data right at your fingertips. Composed of a mixture of symbols and text, regular expressions can be an outlet for creativity, for brilliant programming, and for the elegant solution. Regular Expression Pocket Reference offers an introduction to regular expressions, pattern matching, metacharacters, modes and constructs, and then provides separate sections for each of the language APIs, with complete regex listings including:Supported metacharacters for each language API Regular expression classes and interfaces for Ruby, Java, .NET, and C# Regular expression operators for Perl 5.8 Regular expression module objects and functions for Python Pattern-matching functions for PHP and the vi editor Pattern-matching methods and objects for JavaScript Unicode Support for each of the languages With plenty of examples and other resources, Regular Expression Pocket Reference summarizes the complex rules for performing this critical text-processing function, and presents this often-confusing topic in a friendly and well-organized format. This guide makes an ideal on-the-job companion.

    "The Rails(TM) 3 Way"is the only comprehensive, authoritative guide to delivering production-quality code with Rails 3. Pioneering Rails expert Obie Fernandez and a team of leading experts illuminate the entire Rails 3 API, along with the idioms, design approaches, and libraries that make developing applications with Rails so powerful. Drawing on their unsurpassed experience and track record, they address the real challenges development teams face, showing how to use Rails 3 to maximize your productivity. Using numerous detailed code examples, the author systematically covers Rails 3 key capabilities and subsystems, making this book a reference that you will turn to again and again. He presents advanced Rails programming techniques that have been proven effective in day-to-day usage on dozens of production Rails systems and offers important insights into behavior-driven development and production considerations such as scalability. Dive deep into the Rails 3 codebase and discover why Rails is designed the way it is--and how to make it do what you want it to do.This book will help youLearn what's new in Rails 3 Increase your productivity as a web application developer Realize the overall joy in programming with Rails Leverage Rails' powerful capabilities for building REST-compliant APIs Drive implementation and protect long-term maintainability using RSpec Design and manipulate your domain layer using Active Record Understand and program complex program flows using Action Controller Master sophisticated URL routing concepts Use Ajax techniques via Rails 3 support for unobtrusive JavaScript Learn to extend Rails with popular gems and plugins, and how to write your own Extend Rails with the best third-party plug-ins and write your own Integrate email services into your applications with Action Mailer Improve application responsiveness with background processing Create your own non-Active Record domain classes using Active Model Master Rails' utility classes and extensions in Active Support

    It starts with core concepts like sending, receiving, routing, and transforming data and then shows readers the entire lifecycle. The book goes in depth on how to test, deal with errors, scale, deploy, and monitor apps and even how to build custom tooling. Written by core developers of Camel and the authors of the first edition, this book distills their experience and practical insights so that readers can tackle integration tasks like a pro.Purchase of the print book includes a free eBook in PDF, Kindle, and ePub formats from Manning Publications.

    I'm going to share that mindset with you — along with hundreds of actionable techniques and proven habits — so you can shortcut those years.Introducing The Effective Engineer — the only book designed specifically for today's software engineers, based on extensive interviews with engineering leaders at top tech companies, and packed with hundreds of techniques to accelerate your career.For two years, I embarked on a quest seeking an answer to one question:How do the most effective engineers make their efforts, their teams, and their careers more successful?I interviewed and collected stories from engineering VPs, directors, managers, and other leaders at today's top software companies: established, household names like Google, Facebook, Twitter, and LinkedIn; rapidly growing mid-sized companies like Dropbox, Square, Box, Airbnb, and Etsy; and startups like Reddit, Stripe, Instagram, and Lyft.These leaders shared stories about the most valuable insights they've learned and the most common and costly mistakes that they've seen engineers — sometimes themselves — make.This is just a small sampling of the hard questions I posed to them:- What engineering qualities correlate with future success?- What have you done that has paid off the highest returns?- What separates the most effective engineers you've worked with from everyone else?- What's the most valuable lesson your team has learned in the past year?- What advice do you give to new engineers on your team? Everyone's story is different, but many of the lessons share common themes.You'll get to hear stories like:- How did Instagram's team of 5 engineers build and support a service that grew to over 40 million users by the time the company was acquired?- How and why did Quora deploy code to production 40 to 50 times per day?- How did the team behind Google Docs become the fastest acquisition to rewrite its software to run on Google's infrastructure?- How does Etsy use continuous experimentation to design features that are guaranteed to increase revenue at launch?- How did Facebook's small infrastructure team effectively operate thousands of database servers?- How did Dropbox go from barely hiring any new engineers to nearly tripling its team size year-over-year? What's more, I've distilled their stories into actionable habits and lessons that you can follow step-by-step to make your career and your team more successful.The skills used by effective engineers are all learnable.And I'll teach them to you. With The Effective Engineer, I'll teach you a unifying framework called leverage — the value produced per unit of time invested — that you can use to identify the activities that produce disproportionate results.Here's a sneak peek at some of the lessons you'll learn. You'll learn how to:- Prioritize the right projects and tasks to increase your impact.- Earn more leeway from your peers and managers on your projects.- Spend less time maintaining and fixing software and more time building and shipping new features.- Produce more accurate software estimates.- Validate your ideas cheaply to reduce wasted work.- Navigate organizational and people-related bottlenecks.- Find the appropriate level of code reviews, testing, abstraction, and technical debt to balance speed and quality.- Shorten your debugging workflow to increase your iteration speed.

    You need to assume it will run in the most hostile environments imaginable -- and design, code, and test accordingly. Writing Secure Code, Second Edition shows you how. This edition draws on the lessons learned and taught throughout Microsoft during the firm s massive 2002 Windows Security Push. It s a huge upgrade to the respected First Edition, with new coverage across the board. Michael Howard and David LeBlanc first help you define what security means to your customers -- and implement a three-pronged strategy for securing design, defaults, and deployment. There s especially useful coverage of threat modeling -- decomposing your application, identifying threats, ranking them, and mitigating them. Then, it s on to in-depth coverage of today s key security issues from the developer s standpoint. Everyone knows buffer overruns are bad: Here s a full chapter on avoiding them. You ll learn how to establish appropriate access controls and default to running with least privilege. There s detailed coverage of overcoming attacks on cryptography (for example, avoiding poor random numbers and bit-flipping attacks). You ll learn countermeasures for virtually every form of user input attack, from malicious database updates to cross-site scripting. We ve just scratched the surface: There are authoritative techniques for securing sockets and RPC, protecting against DOS attacks, building safer .NET applications, reviewing and testing code, adding privacy features, and even writing high-quality security documentation. Following these techniques won t just improve security -- it ll dramatically improve robustness and reliability, too. Bill CamardaBill Camarda is a consultant, writer, and web/multimedia content developer. His 15 books include Special Edition Using Word 2000 and Upgrading & Fixing Networks For Dummies®, Second Edition.

    Now this same team provides the best explanation yet of these new features, and of concurrency in general. Concurrency is no longer a subject for advanced users only. Every Java developer should read this book."--Martin BuchholzJDK Concurrency Czar, Sun Microsystems"For the past 30 years, computer performance has been driven by Moore's Law; from now on, it will be driven by Amdahl's Law. Writing code that effectively exploits multiple processors can be very challenging. Java Concurrency in Practice provides you with the concepts and techniques needed to write safe and scalable Java programs for today's--and tomorrow's--systems."--Doron RajwanResearch Scientist, Intel Corp"This is the book you need if you're writing--or designing, or debugging, or maintaining, or contemplating--multithreaded Java programs. If you've ever had to synchronize a method and you weren't sure why, you owe it to yourself and your users to read this book, cover to cover."--Ted NewardAuthor of Effective Enterprise Java"Brian addresses the fundamental issues and complexities of concurrency with uncommon clarity. This book is a must-read for anyone who uses threads and cares about performance."--Kirk PepperdineCTO, JavaPerformanceTuning.com"This book covers a very deep and subtle topic in a very clear and concise way, making it the perfect Java Concurrency reference manual. Each page is filled with the problems (and solutions!) that programmers struggle with every day. Effectively exploiting concurrency is becoming more and more important now that Moore's Law is delivering more cores but not faster cores, and this book will show you how to do it."--Dr. Cliff ClickSenior Software Engineer, Azul Systems"I have a strong interest in concurrency, and have probably written more thread deadlocks and made more synchronization mistakes than most programmers. Brian's book is the most readable on the topic of threading and concurrency in Java, and deals with this difficult subject with a wonderful hands-on approach. This is a book I am recommending to all my readers of The Java Specialists' Newsletter, because it is interesting, useful, and relevant to the problems facing Java developers today."--Dr. Heinz KabutzThe Java Specialists' Newsletter"I've focused a career on simplifying simple problems, but this book ambitiously and effectively works to simplify a complex but critical subject: concurrency. Java Concurrency in Practice is revolutionary in its approach, smooth and easy in style, and timely in its delivery--it's destined to be a very important book."--Bruce TateAuthor of Beyond Java" Java Concurrency in Practice is an invaluable compilation of threading know-how for Java developers. I found reading this book intellectually exciting, in part because it is an excellent introduction to Java's concurrency API, but mostly because it captures in a thorough and accessible way expert knowledge on threading not easily found elsewhere."--Bill VennersAuthor of Inside the Java Virtual MachineThreads are a fundamental part of the Java platform. As multicore processors become the norm, using concurrency effectively becomes essential for building high-performance applications. Java SE 5 and 6 are a huge step forward for the development of concurrent applications, with improvements to the Java Virtual Machine to support high-performance, highly scalable concurrent classes and a rich set of new concurrency building blocks. In Java Concurrency in Practice , the creators of these new facilities explain not only how they work and how to use them, but also the motivation and design patterns behind them.However, developing, testing, and debugging multithreaded programs can still be very difficult; it is all too easy to create concurrent programs that appear to work, but fail when it matters most: in production, under heavy load. Java Concurrency in Practice arms readers with both the theoretical underpinnings and concrete techniques for building reliable, scalable, maintainable concurrent applications. Rather than simply offering an inventory of concurrency APIs and mechanisms, it provides design rules, patterns, and mental models that make it easier to build concurrent programs that are both correct and performant.This book covers:Basic concepts of concurrency and thread safety Techniques for building and composing thread-safe classes Using the concurrency building blocks in java.util.concurrent Performance optimization dos and don'ts Testing concurrent programs Advanced topics such as atomic variables, nonblocking algorithms, and the Java Memory Model

    With the many upgraded features, C# is more expressive than ever. However, an in depth understanding is required to get the most out of the language.C# in Depth, Second Edition is a thoroughly revised, up-to-date book that covers the new features of C# 4 as well as Code Contracts. In it, you'll see the subtleties of C# programming in action, learning how to work with high-value features that you'll be glad to have in your toolkit. The book helps readers avoid hidden pitfalls of C# programming by understanding "behind the scenes" issues.Purchase of the print book comes with an offer of a free PDF, ePub, and Kindle eBook from Manning. Also available is all code from the book.

    Database performance has become a major headache, and most IT departments believe that developers should provide simple SQL code to solve immediate problems and let DBAs tune any bad SQL later.In The Art of SQL, author and SQL expert Stephane Faroult argues that this safe approach only leads to disaster. His insightful book, named after Art of War by Sun Tzu, contends that writing quick inefficient code is sweeping the dirt under the rug. SQL code may run for 5 to 10 years, surviving several major releases of the database management system and on several generations of hardware. The code must be fast and sound from the start, and that requires a firm understanding of SQL and relational theory.The Art of SQL offers best practices that teach experienced SQL users to focus on strategy rather than specifics. Faroult's approach takes a page from Sun Tzu's classic treatise by viewing database design as a military campaign. You need knowledge, skills, and talent. Talent can't be taught, but every strategist from Sun Tzu to modern-day generals believed that it can be nurtured through the experience of others. They passed on their experience acquired in the field through basic principles that served as guiding stars amid the sound and fury of battle. This is what Faroult does with SQL.Like a successful battle plan, good architectural choices are based on contingencies. What if the volume of this or that table increases unexpectedly? What if, following a merger, the number of users doubles? What if you want to keep several years of data online? Faroult's way of looking at SQL performance may be unconventional and unique, but he's deadly serious about writing good SQL and using SQL well. The Art of SQL is not a cookbook, listing problems and giving recipes. The aim is to get you-and your manager-to raise good questions.

    These 30 short chapters each deal with a particular programming problem drawn from sources as diverse as games and puzzles, intriguing combinatorial tasks, and more familiar areas such as data compression and string matching. Each pearl starts with the statement of the problem expressed using the functional programming language Haskell, a powerful yet succinct language for capturing algorithmic ideas clearly and simply. The novel aspect of the book is that each solution is calculated from an initial formulation of the problem in Haskell by appealing to the laws of functional programming. Pearls of Functional Algorithm Design will appeal to the aspiring functional programmer, students and teachers interested in the principles of algorithm design, and anyone seeking to master the techniques of reasoning about programs in an equational style.