Part Ⅰ．Foundations 3
1．Challenges and Principles 3
Why Infrastructure as Code? 3
What Is Infrastructure as Code? 5
Goals of Infrastructure as Code 5
Challenges with Dynamic Infrastructure 6
Server Sprawl 6
Configuration Drift 7
Snowflake Servers 7
Fragile Infrastructure 8
Automation Fear 9
Erosion 10
Principles of Infrastructure as Code 10
Systems Can Be Easily Reproduced 10
Systems Are Disposable 11
Systems Are Consistent 12
Processes Are Repeatable 12
Design Is Always Changing 13
Practices 13
Use Definition Files 14
Self-Documented Systems and Processes 14
Version All the Things 15
Continuously Test Systems and Processes 16
Small Changes Rather Than Batches 16
Keep Services Available Continuously 17
Antifragility：Beyond“Robust” 17
The Secret Ingredient of Antifragile IT Systems 18
Conclusion 19
What’s Next? 19
2．Dynamic Infrastructure Platforms 21
What Is a Dynamic Infrastructure Platform? 21
Requirements for a Dynamic Infrastructure Platform 22
Programmable 23
On-Demand 24
Self-Service 25
Infrastructure Resources Provided by the Platform 25
Compute Resources 26
Storage Resources 26
Network Resources 28
Types of Dynamic Infrastructure Platforms 30
Public IaaS Cloud 30
Community IaaS Cloud 30
Private IaaS Cloud 30
Antipattern：Hand-Cranked Cloud 31
Hybrid and Mixed Cloud Options 32
Bare-Metal Clouds 32
Deciding on a Dynamic Infrastructure Platform 34
Public or Private? 34
Cloud Portability 37
Mechanical Sympathy with the Cloud and Virtualization 39
Conclusion 40
3．Infrastructure Definition Tools 41
Choosing Tools for Infrastructure as Code 42
Requirement：Scriptable Interface 42
Requirement：Unattended Mode for Command-Line Tools 42
Requirement：Support for Unattended Execution 43
Requirement：Externalized Configuration 45
Configuration Definition Files 48
Reusability with Configuration Definitions 49
Working with Infrastructure Definition Tools 50
Provisioning Infrastructure with Procedural Scripts 51
Defining Infrastructure Declaratively 53
Using Infrastructure Definition Tools 54
Configuring Servers 54
Configuration Registries 55
Lightweight Configuration Registries 56
Is a Configuration Registry a CMDB? 57
The CMDB Audit and Fix Antipattern 58
The Infrastructure-as-Code Approach to CMDB 59
Conclusion 59
4．Server Configuration Tools 61
Goals for Automated Server Management 62
Tools for Different Server Management Functions 62
Tools for Creating Servers 63
Tools for Configuring Servers 64
Tools for Packaging Server Templates 65
Tools for Running Commands on Servers 66
Using Configuration from a Central Registry 68
Server Change Management Models 69
Ad Hoc Change Management 69
Configuration Synchronization 69
Immutable Infrastructure 70
Containerized Services 70
Containers 70
Managing Ruby Applications with and without Containers 72
Are Containers Virtual Machines? 73
Using Containers Rather than Virtual Machines 74
Running Containers 75
Security and Containers 76
Conclusion 78
5．General Infrastructure Services 81
Considerations for Infrastructure Services and Tools 81
Prefer Tools with Externalized Configuration 83
Prefer Tools That Assume Infrastructure Is Dynamic 84
Prefer Products with Cloud-Compatible Licensing 84
Prefer Products That Support Loose Coupling 85
Sharing a Service Between Teams 85
Service Instance Templates 86
Monitoring：Alerting,Metrics,and Logging 87
Alerting：Tell Me When Something Is Wrong 87
Metrics：Collect and Analyze Data 89
Log Aggregation and Analysis 89
Service Discovery 90
Server-Side Service Discovery Pattern 91
Client-Side Service Discovery Pattern 91
Distributed Process Management 91
Orchestrating Processes with Server Roles 92
Orchestrating Processes with Containers 92
Scheduling Short Jobs 92
Container Orchestration Tools 92
Software Deployment 93
Deployment Pipeline Software 93
Packaging Software 94
Conclusion 96
Part Ⅱ．Patterns 99
6．Patterns for Provisioning Servers 99
Server Provisioning 100
A Server’s Life 100
What Goes onto a Server 105
Types of Things on a Server 105
Server Roles 107
Patterns for Creating Servers 108
Antipattern：Handcrafted Server 109
Practice：Wrap Server Creation Options in a Script 110
Antipattern：Hot Cloned Server 111
Pattern：Server Template 111
Antipattern：Snowflake Factory 112
Patterns for Bootstrapping New Servers 112
Pushing to Bootstrap 113
Pulling to Bootstrap 113
Practice：Smoke Test Every New Server Instance 114
Conclusion 115
7．Patterns for Managing Server Templates 117
Stock Templates：Can’t Someone Else Do It? 117
Provisioning Servers Using Templates 118
Provisioning at Creation Time 118
Provisioning in the Template 119
Balancing Provisioning Across Template and Creation 120
The Process for Building a Server Template 121
Creating Templates for Multiple Platforms 122
Origin Images 123
Antipattern：Hot Cloned Server Template 123
Baking a Template from an OS Installation Image 123
Baking a Template from a Stock Image 124
Building a Template from a Unikernel 125
Customizing a Server Template without Booting It 125
Updating Server Templates 126
Reheating a Template 126
Baking a Fresh Template 126
Versioning Server Templates 126
Building Templates for Roles 129
Pattern：Layered Template 129
Sharing Base Scripts for Templates 130
Automating Server Template Management 131
Customizing Servers Before Baking 131
Practice：Automatically Test Server Templates 132
Conclusion 132
8．Patterns for Updating and Changing Servers 133
Models for Server Change Management 134
Ad Hoc Change Management 134
Continuous Configuration Synchronization 134
Immutable Servers 135
Containerized Servers 136
General Patterns and Practices 136
Practice：Minimize Server Templates 136
Practice：Replace Servers When the Server Template Changes 137
Pattern：Phoenix Servers 137
Patterns and Practices for Continuous Deployment 138
Pattern：Masterless Configuration Management 139
Practice：Apply Cron 139
Continuous Synchronization Flow 140
The Unconfigured Country 141
Patterns and Practices for Immutable Servers 143
Server Image as Artifact 144
Simplifying Confirmation Management Tooling with Immutable Servers 144
Immutable Server Flow 144
Bootstrap Configuration with Immutable Servers 145
Transactional Server Updates 147
Practices for Managing Configuration Definitions 148
Practice：Keep Configuration Definitions Minimal 148
Organizing Definitions 149
Practice：Use Test-Driven Development（TDD）to Drive Good Design 149
Conclusion 150
9．Patterns for Defining Infrastructure 151
Environments 152
Antipattern：Handcrafted Infrastructure 153
Defining Infrastructure Stacks as Code 153
Antipattern：Per-Environment Definition Files 155
Pattern：Reusable Definition Files 155
Practice：Test and Promote Stack Definitions 157
Self-Service Environments 158
Organizing Infrastructure 158
Antipattern：Monolithic Stack 158
Avoid“Lift and Shift”When Migrating Infrastructure 160
Dividing an Application Environment into Multiple Stacks 160
Managing Configuration Parameters Between Stacks 162
Sharing Infrastructure Elements 164
Practice：Manage Application Code and Infrastructure Code Together 166
Approaches to Sharing Definitions 167
Practice：Align Infrastructure Design with the Scope of Change 168
Example：An Infrastructure Design for Microservices 169
Running Definition Tools 174
Conclusion 175
Part Ⅲ．Practices 179
10．Software Engineering Practices for Infrastructure 179
System Quality 180
Poor-Quality Systems Are Difficult to Change 180
High-Quality Systems Are Easier and Safer to Change 181
Infrastructure Quality Through Code 181
Fast Feedback 181
VCS for Infrastructure Management 182
What to Manage in a VCS 182
Continuous Integration（CI） 183
Continuously Testing Branches Is Not Continuous Integration 183
Who Broke the Build? 185
Ignoring Tests That Fail 186
CI for Infrastructure 187
Continuous Delivery（CD） 187
The Problem with the Integration Phase 187
Deployment Pipelines and Change Pipelines 188
Continuous Delivery Is Not Continuous Deployment 189
Code Quality 190
Clean Code 190
Practice：Manage Technical Debt 191
Managing Major Infrastructure Changes 192
Feature Toggles 193
Conclusion 194
11．Testing Infrastructure Changes 195
The Agile Approach to Testing 196
Automating Tests for Fast Feedback 197
Organically Building a Test Suite 197
Structuring the Test Suite：The Test Pyramid 198
Avoiding an Unbalanced Test Suite 199
Practice：Test at the Lowest Level Possible 200
Practice：Only Implement the Layers You Need 201
Practice：Prune the Test Suite Often 202
Practice：Continuously Review Testing Effectiveness 202
Implementing a Balanced Test Suite 202
Low-Level Testing 204
Mid-Level Testing 206
Higher-Level Tests 209
Testing Operational Quality 211
Managing Test Code 212
Practice：Keep Test Code with the Code It Tests 212
Anti-Pattern：Reflective Tests 213
Techniques to Isolate Components for Testing 213
Refactoring Components so They Can Be Isolated 215
Managing External Dependencies 215
Test Setup 216
Roles and Workflow for Testing 218
Principle：People Should Write Tests for What They Build 219
The Test-Writing Habit 219
Principle：Everyone Should Have Access to the Testing Tools 219
The Value of a Quality Analyst 220
Test-Driven Development（TDD） 221
Conclusion 222
12．Change Management Pipelines for Infrastructure 223
Benefits of a Change Management Pipeline 225
Guidelines for Designing Pipelines 226
Ensure Consistency Across Stages 226
Get Immediate Feedback for Every Change 227
Run Automated Stages Before Manual Stages 227
Get Production-Like Sooner Rather Than Later 228
Basic Pipeline Designs 229
The Local Development Stage 229
The Build Stage 230
Publishing a Configuration Artifact 231
Automated Testing Stages 233
Manual Validation Stages 234
Apply to Live 235
The Rhythm of the Pipeline 235
Practices for Using a Pipeline 236
Practice：Prove Production Readiness for Every Change 237
Practice：Start Every Change from the Beginning of the Pipeline 237
Practice：Stop the Line on Any Failure 238
Scaling Pipelines to More Complex Systems 238
Pattern：Fan-In Pipelines 239
Practice：Keep Pipelines Short 243
Practice：Decouple Pipelines 243
Integration Models 243
Techniques for Handling Dependencies Between Components 246
Pattern：Library Dependency 246
Pattern：Self-Provisioned Service Instance 248
Providing Pre-Release Library Builds 248
Providing Test Instances of a Service to Consumers 249
Using Test Instances of a Service as a Consumer 250
Practices for Managing Interfaces Between Components 252
Practice：Ensure Backward Compatibility of Interfaces 252
Practice：Decouple Deploying from Releasing 253
Practice：Use Version Tolerance 253
Practice：Provide Test Doubles 254
Practice：Test the Provider with Contract Tests 254
Practice：Test with a Reference Consumer 255
Practice：Smoke Test the Provider Interface 255
Practice：Run Consumer-Driven Contract（CDC）Tests 255
Conclusion 256
13．Workflow for the Infrastructure Team 257
Automate Anything That Moves 258
Make the Change Manually 259
Ad Hoc Automation 259
Autonomic Automation 260
Autonomic Automation Workflow 261
Using a Local Sandbox 262
Using Local Virtualization for a Sandbox 263
Example Workflow with Local Testing 265
Using the Virtualization Platform for Sandboxes 266
Codebase Organization Patterns 267
Antipattern：Branch-Based Codebases 268
Pattern：One Trunk per Component 269
Pattern：Single Trunk 269
Workflow Effectiveness 269
Expediting Changes 270
Code Reviews 271
Fitting Governance into the Workflow 271
Conclusion 273
14．Continuity with Dynamic Infrastructure 275
Service Continuity 276
True Availability 276
Using Dynamic Server Pools for Recovery 277
Software Design for Dynamic Infrastructure 279
Compartmentalizing for Continuity 281
Zero-Downtime Changes 282
Pattern：Blue-Green Replacement 282
Pattern：Phoenix Replacement 283
Practice：Reduce the Scope of Replacement 284
Pattern：Canary Replacement 285
Routing Traffic for Zero-Downtime Replacements 286
Zero-Downtime Changes with Data 288
Data Continuity 289
Replicating Data Redundantly 290
Regenerating Data 290
Delegating Data Persistence 291
Backing Up to Persistent Storage 291
Disaster Recovery 292
Continuous Disaster Recovery 293
The DR Plan：Planning for Disaster 294
Practice：Prefer Rebuilding Things over Cold Standby 295
Continuous Monitoring through the Pipeline 296
Security 298
Automatically Papering over Compromises 298
Reliable Updates as a Defense 299
Provenance of Packages 299
Automated Hardening 301
Automating Security Validation in the Pipeline 302
The Change Pipeline as a Vulnerability 302
Managing Security Risks with Cloud Accounts 303
Conclusion 306
15．Organizing for Infrastructure as Code 307
Evolutionary Architecture 308
Learning under Fire 309
Start with a Trailblazer Pipeline 310
Measuring Effectiveness 311
Agree on Desired Outcomes First 311
Choose Metrics that Help the Team 312
Track and Improve Cycle Time 313
Use Kanban to Make Work Visible 315
Retrospectives and Post-Mortems 316
Organize to Empower Users 317
Pitfalls of the Divided Function Model 317
Adopt a Self-Service Model 319
Take Full Responsibility：You Build It,You Run It 320
Organizing into Cross-Functional Teams 321
Governance through Continuous Change Management 322
Provide Solid Building Blocks 323
Prove Operational Readiness in the Pipeline 323
Sharing Ownership of Operational Quality 324
Review and Audit Automated Processes 324
Optimize for Time to Detect and Fix Problems 325
Conclusion：It’s Never Finished 325
Index 327