DeveloperBlogs

Internal Tooling Innovation at Codeweavers

Two decades ago, our deployment pipeline needed a brain, or at least a nervous system. Out of that need, Gaffa was born.  

Originally built as a deployment visualisation tool, Gaffa has since evolved into a core piece of infrastructure that orchestrates hundreds of builds and deployments each day from our Engineering Team. While many businesses today rely on off-the-shelf DevOps tooling, Gaffa represents something else: what happens when a tool is grown from scratch, shaped by years of practical need, and improved through relentless iteration. 

This is the story of how we engineered Gaffa, from a passive observer into a proactive engine. 

Why we built Gaffa 

In the early 2000s, deployment pipelines were black boxes, especially for teams working with hybrid stacks and complex testing requirements. We didn’t just want logs; we wanted visibility, control, accountability, and the ability to evolve our release process without rebuilding everything from scratch each time. 

So, we built Gaffa, initially with the simple aim of visualising deployments to our local, testing and live environments as a deployment dashboard, allowing our developers to see, at a glance, the state of their changes. But like any useful tool, it grew. 

Technical evolution  

Over time, what began as a simple visualisation layer steadily evolved into a robust deployment engine, shaped by real-world demands and developer feedback. We had several key phases in its development: 

Phase 1: Visualisation and monitoring  

Initially, Gaffa was designed to provide visibility into our deployment processes. It displayed the status of deployments and highlighted failures across different environments. While it didn’t control deployments, it offered much-needed insight at a time when most processes were opaque. Gaffa integrated directly with our internal tools, and though its alerting and monitoring features were minimal, they immediately improved developer confidence in the release process. 

Phase 2: Deployment orchestration 

As our needs grew, Gaffa transitioned from a passive observer to an active participant in the deployment flow. It began orchestrating code integration, managing job queues, and automating environment promotions. The system formalised the path from build to release, moving code through a defined sequence of environments, from Build to Local, then to Demo and finally Live. Promotion logic was codified to enforce safe transitions, prevent regressions, and support rollback conditions. 

Phase 3: Automated testing and feedback loops 

To improve reliability, we began incorporating automated testing directly into the pipeline. It supported a range of test types, including compilation checks, end-to-end tests, and regression suites. We also introduced canary testing, enabling small-scale live deployments before a full rollout. Gaffa provided immediate feedback on test results and included smart retry mechanisms. 

Phase 4: Scale and performance optimisation  

Today, Gaffa handles approximately 200 deployments per day, ranging from minor UI changes to full-scale feature releases. It was optimised to balance speed and safety, ensuring rapid iteration without compromising system stability. Its architecture supports parallel execution and is resilient under high load, enabling our engineering teams to move quickly and confidently. 

Why we kept building 

Every engineering team reaches a crossroads at some point: should we build, or should we buy? For us, the answer time and again was to keep building. Not out of nostalgia or fear of change, but because Gaffa consistently proved to be the right tool for our evolving needs. From day one, it offered tangible advantages that off-the-shelf solutions couldn’t replicate without compromising flexibility or control. It gave us full ownership over how releases worked, enabled early and automated testing throughout the pipeline, and integrated deeply with our internal ecosystem. Rather than forcing our workflows to conform to a generic tool, we shaped Gaffa to mirror how we actually build, test, and ship software. 

Over the past two decades, Gaffa has evolved from a simple visualisation tool into the backbone of our deployment process, adapting with us through every shift in technology, team structure, and product direction. It’s a testament to what’s possible when engineers are empowered to build what they need, iterate continuously, and refine over time. While the tooling landscape around us has changed dramatically, the core principles that guided Gaffa’s development remain just as relevant: visibility, control, flexibility, and trust in the deployment pipeline. 

Written by Matt Sutton, Senior Lead Software Engineer

Codeweavers

Read more by Codeweavers