Ideal for processing large datasets, the Apache Hadoop framework is an open source implementation of the MapReduce algorithm on which Google built its empire. This comprehensive resource demonstrates how to use Hadoop to build reliable, scalable, distributed systems: programmers will find details for analyzing large datasets, and administrators will learn how to set up and run Hadoop clusters. Complete with case studies that illustrate how Hadoop solves specific problems, this book helps you:Use the Hadoop Distributed File System (HDFS) for storing large datasets, and run distributed computations over those datasets using MapReduce Become familiar with Hadoop's data and I/O building blocks for compression, data integrity, serialization, and persistence Discover common pitfalls and advanced features for writing real-world MapReduce programs Design, build, and administer a dedicated Hadoop cluster, or run Hadoop in the cloud Use Pig, a high-level query language for large-scale data processing Take advantage of HBase, Hadoop's database for structured and semi-structured data Learn ZooKeeper, a toolkit of coordination primitives for building distributed systems If you have lots of data -- whether it's gigabytes or petabytes -- Hadoop is the perfect solution. Hadoop: The Definitive Guide is the most thorough book available on the subject. "Now you have the opportunity to learn about Hadoop from a master-not only of the technology, but also of common sense and plain talk." -- Doug Cutting, Hadoop Founder, Yahoo!

    However, data structures for these languages do not always translate well to functional languages such as Standard ML, Haskell, or Scheme. This book describes data structures from the point of view of functional languages, with examples, and presents design techniques that allow programmers to develop their own functional data structures. The author includes both classical data structures, such as red-black trees and binomial queues, and a host of new data structures developed exclusively for functional languages. All source code is given in Standard ML and Haskell, and most of the programs are easily adaptable to other functional languages. This handy reference for professional programmers working with functional languages can also be used as a tutorial or for self-study.

    You'll learn how to:Read and write files efficiently Use signals, clocks, and timers Create processes and execute programs Write secure programs Write multithreaded programs using POSIX threads Build and use shared libraries Perform interprocess communication using pipes, message queues, shared memory, and semaphores Write network applications with the sockets API While The Linux Programming Interface covers a wealth of Linux-specific features, including epoll, inotify, and the /proc file system, its emphasis on UNIX standards (POSIX.1-2001/SUSv3 and POSIX.1-2008/SUSv4) makes it equally valuable to programmers working on other UNIX platforms.The Linux Programming Interface is the most comprehensive single-volume work on the Linux and UNIX programming interface, and a book that's destined to become a new classic.Praise for The Linux Programming Interface "If I had to choose a single book to sit next to my machine when writing software for Linux, this would be it." —Martin Landers, Software Engineer, Google "This book, with its detailed descriptions and examples, contains everything you need to understand the details and nuances of the low-level programming APIs in Linux . . . no matter what the level of reader, there will be something to be learnt from this book." —Mel Gorman, Author of Understanding the Linux Virtual Memory Manager "Michael Kerrisk has not only written a great book about Linux programming and how it relates to various standards, but has also taken care that bugs he noticed got fixed and the man pages were (greatly) improved. In all three ways, he has made Linux programming easier. The in-depth treatment of topics in The Linux Programming Interface . . . makes it a must-have reference for both new and experienced Linux programmers." —Andreas Jaeger, Program Manager, openSUSE, Novell "Michael's inexhaustible determination to get his information right, and to express it clearly and concisely, has resulted in a strong reference source for programmers. While this work is targeted at Linux programmers, it will be of value to any programmer working in the UNIX/POSIX ecosystem." —David Butenhof, Author of Programming with POSIX Threads and Contributor to the POSIX and UNIX Standards ". . . a very thorough—yet easy to read—explanation of UNIX system and network programming, with an emphasis on Linux systems. It's certainly a book I'd recommend to anybody wanting to get into UNIX programming (in general) or to experienced UNIX programmers wanting to know 'what's new' in the popular GNU/Linux system." —Fernando Gont, Network Security Researcher, IETF Participant, and RFC Author ". . . encyclopedic in the breadth and depth of its coverage, and textbook-like in its wealth of worked examples and exercises. Each topic is clearly and comprehensively covered, from theory to hands-on working code. Professionals, students, educators, this is the Linux/UNIX reference that you have been waiting for." —Anthony Robins, Associate Professor of Computer Science, The University of Otago "I've been very impressed by the precision, the quality and the level of detail Michael Kerrisk put in his book. He is a great expert of Linux system calls and lets us share his knowledge and understanding of the Linux APIs." —Christophe Blaess, Author of Programmation systeme en C sous Linux ". . . an essential resource for the serious or professional Linux and UNIX systems programmer. Michael Kerrisk covers the use of all the key APIs across both the Linux and UNIX system interfaces with clear descriptions and tutorial examples and stresses the importance and benefits of following standards such as the Single UNIX Specification and POSIX 1003.1." —Andrew Josey, Director, Standards, The Open Group, and Chair of the POSIX 1003.1 Working Group "What could be better than an encyclopedic reference to the Linux system, from the standpoint of the system programmer, written by none other than the maintainer of the man pages himself? The Linux Programming Interface is comprehensive and detailed. I firmly expect it to become an indispensable addition to my programming bookshelf." —Bill Gallmeister, Author of POSIX.4 Programmer's Guide: Programming for the Real World ". . . the most complete and up-to-date book about Linux and UNIX system programming. If you're new to Linux system programming, if you're a UNIX veteran focused on portability while interested in learning the Linux way, or if you're simply looking for an excellent reference about the Linux programming interface, then Michael Kerrisk's book is definitely the companion you want on your bookshelf." —Loic Domaigne, Chief Software Architect (Embedded), Corpuls.com

    They are now standard features in a wide range of languages and popular tools, including Perl, Python, Ruby, Java, VB.NET and C# (and any language using the .NET Framework), PHP, and MySQL.If you don't use regular expressions yet, you will discover in this book a whole new world of mastery over your data. If you already use them, you'll appreciate this book's unprecedented detail and breadth of coverage. If you think you know all you need to know about regularexpressions, this book is a stunning eye-opener.As this book shows, a command of regular expressions is an invaluable skill. Regular expressions allow you to code complex and subtle text processing that you never imagined could be automated. Regular expressions can save you time and aggravation. They can be used to craft elegant solutions to a wide range of problems. Once you've mastered regular expressions, they'll become an invaluable part of your toolkit. You will wonder how you ever got by without them.Yet despite their wide availability, flexibility, and unparalleled power, regular expressions are frequently underutilized. Yet what is power in the hands of an expert can be fraught with peril for the unwary. Mastering Regular Expressions will help you navigate the minefield to becoming an expert and help you optimize your use of regular expressions.Mastering Regular Expressions, Third Edition, now includes a full chapter devoted to PHP and its powerful and expressive suite of regular expression functions, in addition to enhanced PHP coverage in the central "core" chapters. Furthermore, this edition has been updated throughout to reflect advances in other languages, including expanded in-depth coverage of Sun's java.util.regex package, which has emerged as the standard Java regex implementation.Topics include:A comparison of features among different versions of many languages and toolsHow the regular expression engine worksOptimization (major savings available here!)Matching just what you want, but not what you don't wantSections and chapters on individual languagesWritten in the lucid, entertaining tone that makes a complex, dry topic become crystal-clear to programmers, and sprinkled with solutions to complex real-world problems, Mastering Regular Expressions, Third Edition offers a wealth information that you can put to immediateuse.Reviews of this new edition and the second edition: "There isn't a better (or more useful) book available on regular expressions."--Zak Greant, Managing Director, eZ Systems"A real tour-de-force of a book which not only covers the mechanics of regexes in extraordinary detail but also talks about efficiency and the use of regexes in Perl, Java, and .NET...If you use regular expressions as part of your professional work (even if you already have a good book on whatever language you're programming in) I would strongly recommend this book to you."--Dr. Chris Brown, Linux Format"The author does an outstanding job leading the reader from regexnovice to master. The book is extremely easy to read and chock full ofuseful and relevant examples...Regular expressions are valuable toolsthat every developer should have in their toolbox. Mastering RegularExpressions is the definitive guide to the subject, and an outstandingresource that belongs on every programmer's bookshelf. Ten out of TenHorseshoes."--Jason Menard, Java Ranch

    How can you work with it efficiently? Recently updated for Spark 1.3, this book introduces Apache Spark, the open source cluster computing system that makes data analytics fast to write and fast to run. With Spark, you can tackle big datasets quickly through simple APIs in Python, Java, and Scala. This edition includes new information on Spark SQL, Spark Streaming, setup, and Maven coordinates. Written by the developers of Spark, this book will have data scientists and engineers up and running in no time. You’ll learn how to express parallel jobs with just a few lines of code, and cover applications from simple batch jobs to stream processing and machine learning. Quickly dive into Spark capabilities such as distributed datasets, in-memory caching, and the interactive shell Leverage Spark’s powerful built-in libraries, including Spark SQL, Spark Streaming, and MLlib Use one programming paradigm instead of mixing and matching tools like Hive, Hadoop, Mahout, and Storm Learn how to deploy interactive, batch, and streaming applications Connect to data sources including HDFS, Hive, JSON, and S3 Master advanced topics like data partitioning and shared variables

    Now that machine learning is thriving, even programmers who know close to nothing about this technology can use simple, efficient tools to implement programs capable of learning from data. This practical book shows you how.By using concrete examples, minimal theory, and two production-ready Python frameworks—Scikit-Learn and TensorFlow—author Aurélien Géron helps you gain an intuitive understanding of the concepts and tools for building intelligent systems. You’ll learn how to use a range of techniques, starting with simple Linear Regression and progressing to Deep Neural Networks. If you have some programming experience and you’re ready to code a machine learning project, this guide is for you.This hands-on book shows you how to use:Scikit-Learn, an accessible framework that implements many algorithms efficiently and serves as a great machine learning entry pointTensorFlow, a more complex library for distributed numerical computation, ideal for training and running very large neural networksPractical code examples that you can apply without learning excessive machine learning theory or algorithm details

    This classic book uncovers what interviews are really like at America's top software and computer companies and provides you with the tools to succeed in any situation. The authors take you step-by-step through new problems and complex brainteasers they were asked during recent technical interviews. 50 interview scenarios are presented along with in-depth analysis of the possible solutions. The problem-solving process is clearly illustrated so you'll be able to easily apply what you've learned during crunch time. You'll also find expert tips on what questions to ask, how to approach a problem, and how to recover if you become stuck. All of this will help you ace the interview and get the job you want.What you will learn from this bookTips for effectively completing the job application Ways to prepare for the entire programming interview process How to find the kind of programming job that fits you best Strategies for choosing a solution and what your approach says about you How to improve your interviewing skills so that you can respond to any question or situation Techniques for solving knowledge-based problems, logic puzzles, and programming problems Who this book is for This book is for programmers and developers applying for jobs in the software industry or in IT departments of major corporations.Wrox Beginning guides are crafted to make learning programming languages and technologies easier than you think, providing a structured, tutorial format that will guide you through all the techniques involved.

    With the "You Don’t Know JS" book series, you’ll get a more complete understanding of JavaScript, including trickier parts of the language that many experienced JavaScript programmers simply avoid.The series’ first book, Up & Going, provides the necessary background for those of you with limited programming experience. By learning the basic building blocks of programming, as well as JavaScript’s core mechanisms, you’ll be prepared to dive into the other, more in-depth books in the series—and be well on your way toward true JavaScript.With this book you will: Learn the essential programming building blocks, including operators, types, variables, conditionals, loops, and functions Become familiar with JavaScript's core mechanisms such as values, function closures, this, and prototypes Get an overview of other books in the series—and learn why it’s important to understand all parts of JavaScript

     -Byte, September 1995 I can't begin to tell you how many pleasurable hours of study and recreation they have afforded me! I have pored over them in cars, restaurants, at work, at home... and even at a Little League game when my son wasn't in the line-up. -Charles Long If you think you're a really good programmer... read [Knuth's] Art of Computer Programming... You should definitely send me a resume if you can read the whole thing. -Bill Gates It's always a pleasure when a problem is hard enough that you have to get the Knuths off the shelf. I find that merely opening one has a very useful terrorizing effect on computers. -Jonathan Laventhol This first volume in the series begins with basic programming concepts and techniques, then focuses more particularly on information structures-the representation of information inside a computer, the structural relationships between data elements and how to deal with them efficiently. Elementary applications are given to simulation, numerical methods, symbolic computing, software and system design. Dozens of simple and important algorithms and techniques have been added to those of the previous edition. The section on mathematical preliminaries has been extensively revised to match present trends in research. Ebook (PDF version) produced by Mathematical Sciences Publishers (MSP), http: //msp.org

    It has been winning converts from dynamic language enthusiasts as well as users of traditional compiled languages. The former appreciate the robustness and efficiency that Go's lightweight type system brings to their code; the latter find Go's simplicity and fast tools a refreshing change. Thanks to its well-designed standard libraries and its excellent support for concurrent programming, Go is fast becoming the language of choice for distributed systems. The Go Programming Language is the definitive book on Go for the working programmer. It assumes no prior knowledge of Go, nor any other specific programming language, so you'll find it an accessible guide whether you come from JavaScript, Ruby, Python, Java, or C++. The book will quickly get you started using Go effectively from the beginning, and by the end, you will know how to use it well to write clear, idiomatic and efficient programs to solve real-world problems. You'll understand not just how to use its standard libraries, but how they work, and how to apply the same design techniques to your own projects. The earlier chapters will introduce you to the basic concepts of Go programming---numbers, strings, functions---while at the same time presenting important computer science concepts like recursion, and useful examples of graphics, UTF-8, and error handling. The chapters on methods and interfaces will show you a new way to think about object-oriented programming; the chapter on concurrency explains why concurrency is so important in modern programming, and how Go helps you handle it well. You'll also learn about Go's pragmatic but effective approach to testing; how to build, test, and manage projects using the go tool, and the art of metaprogramming using reflection. The book contains hundreds of interesting and practical examples that cover the whole language and a wide range of applications. The code samples from the book are available for download from gopl.io.

    I'm going to share that mindset with you — along with hundreds of actionable techniques and proven habits — so you can shortcut those years.Introducing The Effective Engineer — the only book designed specifically for today's software engineers, based on extensive interviews with engineering leaders at top tech companies, and packed with hundreds of techniques to accelerate your career.For two years, I embarked on a quest seeking an answer to one question:How do the most effective engineers make their efforts, their teams, and their careers more successful?I interviewed and collected stories from engineering VPs, directors, managers, and other leaders at today's top software companies: established, household names like Google, Facebook, Twitter, and LinkedIn; rapidly growing mid-sized companies like Dropbox, Square, Box, Airbnb, and Etsy; and startups like Reddit, Stripe, Instagram, and Lyft.These leaders shared stories about the most valuable insights they've learned and the most common and costly mistakes that they've seen engineers — sometimes themselves — make.This is just a small sampling of the hard questions I posed to them:- What engineering qualities correlate with future success?- What have you done that has paid off the highest returns?- What separates the most effective engineers you've worked with from everyone else?- What's the most valuable lesson your team has learned in the past year?- What advice do you give to new engineers on your team? Everyone's story is different, but many of the lessons share common themes.You'll get to hear stories like:- How did Instagram's team of 5 engineers build and support a service that grew to over 40 million users by the time the company was acquired?- How and why did Quora deploy code to production 40 to 50 times per day?- How did the team behind Google Docs become the fastest acquisition to rewrite its software to run on Google's infrastructure?- How does Etsy use continuous experimentation to design features that are guaranteed to increase revenue at launch?- How did Facebook's small infrastructure team effectively operate thousands of database servers?- How did Dropbox go from barely hiring any new engineers to nearly tripling its team size year-over-year? What's more, I've distilled their stories into actionable habits and lessons that you can follow step-by-step to make your career and your team more successful.The skills used by effective engineers are all learnable.And I'll teach them to you. With The Effective Engineer, I'll teach you a unifying framework called leverage — the value produced per unit of time invested — that you can use to identify the activities that produce disproportionate results.Here's a sneak peek at some of the lessons you'll learn. You'll learn how to:- Prioritize the right projects and tasks to increase your impact.- Earn more leeway from your peers and managers on your projects.- Spend less time maintaining and fixing software and more time building and shipping new features.- Produce more accurate software estimates.- Validate your ideas cheaply to reduce wasted work.- Navigate organizational and people-related bottlenecks.- Find the appropriate level of code reviews, testing, abstraction, and technical debt to balance speed and quality.- Shorten your debugging workflow to increase your iteration speed.

    You'll learn how to use Haskell in a variety of practical ways, from short scripts to large and demanding applications. Real World Haskell takes you through the basics of functional programming at a brisk pace, and then helps you increase your understanding of Haskell in real-world issues like I/O, performance, dealing with data, concurrency, and more as you move through each chapter. With this book, you will:Understand the differences between procedural and functional programming Learn the features of Haskell, and how to use it to develop useful programs Interact with filesystems, databases, and network services Write solid code with automated tests, code coverage, and error handling Harness the power of multicore systems via concurrent and parallel programming You'll find plenty of hands-on exercises, along with examples of real Haskell programs that you can modify, compile, and run. Whether or not you've used a functional language before, if you want to understand why Haskell is coming into its own as a practical language in so many major organizations, Real World Haskell is the best place to start.

    It guides you step by step from simple tests to tests that are maintainable, readable, and trustworthy. It covers advanced subjects like mocks, stubs, and frameworks such as Typemock Isolator and Rhino Mocks. And you'll learn about advanced test patterns and organization, working with legacy code and even untestable code. The book discusses tools you need when testing databases and other technologies. It's written for .NET developers but others will also benefit from this book.Purchase of the print book comes with an offer of a free PDF, ePub, and Kindle eBook from Manning. Also available is all code from the book.Table of ContentsThe basics of unit testingA first unit testUsing stubs to break dependenciesInteraction testing using mock objectsIsolation (mock object) frameworksTest hierarchies and organizationThe pillars of good testsIntegrating unit testing into the organizationWorking with legacy code

    What sets this book apart is its focus on learning. The authors have made the content of OOAD accessible, usable for the practitioner." Ivar Jacobson, Ivar Jacobson Consulting"I just finished reading HF OOA&D and I loved it! The thing I liked most about this book was its focus on why we do OOA&D-to write great software!" Kyle Brown, Distinguished Engineer, IBM"Hidden behind the funny pictures and crazy fonts is a serious, intelligent, extremely well-crafted presentation of OO Analysis and Design. As I read the book, I felt like I was looking over the shoulder of an expert designer who was explaining to me what issues were important at each step, and why." Edward Sciore, Associate Professor, Computer Science Department, Boston College Tired of reading Object Oriented Analysis and Design books that only makes sense after you're an expert? You've heard OOA&D can help you write great software every time-software that makes your boss happy, your customers satisfied and gives you more time to do what makes you happy.But how?Head First Object-Oriented Analysis & Design shows you how to analyze, design, and write serious object-oriented software: software that's easy to reuse, maintain, and extend; software that doesn't hurt your head; software that lets you add new features without breaking the old ones. Inside you will learn how to:Use OO principles like encapsulation and delegation to build applications that are flexible Apply the Open-Closed Principle (OCP) and the Single Responsibility Principle (SRP) to promote reuse of your code Leverage the power of design patterns to solve your problems more efficiently Use UML, use cases, and diagrams to ensure that all stakeholders are communicating clearly to help you deliver the right software that meets everyone's needs.By exploiting how your brain works, Head First Object-Oriented Analysis & Design compresses the time it takes to learn and retain complex information. Expect to have fun, expect to learn, expect to be writing great software consistently by the time you're finished reading this!

    But many organizations adopting these technologies have found that it only leads to a faster-growing sprawl of unmanageable systems. This is where infrastructure as code can help. With this practical guide, author Kief Morris of ThoughtWorks shows you how to effectively use principles, practices, and patterns pioneered through the DevOps movement to manage cloud age infrastructure.Ideal for system administrators, infrastructure engineers, team leads, and architects, this book demonstrates various tools, techniques, and patterns you can use to implement infrastructure as code. In three parts, you'll learn about the platforms and tooling involved in creating and configuring infrastructure elements, patterns for using these tools, and practices for making infrastructure as code work in your environment.Examine the pitfalls that organizations fall into when adopting the new generation of infrastructure technologiesUnderstand the capabilities and service models of dynamic infrastructure platformsLearn about tools that provide, provision, and configure core infrastructure resourcesExplore services and tools for managing a dynamic infrastructureLearn specific patterns and practices for provisioning servers, building server templates, and updating running servers