Engineering·October 2024·9 min read

Strategies for Successfully Managing Large-Scale Web Development Projects

Large-scale web projects don't fail because teams skip stand-ups. They fail because nobody drew a hard line around environments, ownership, and release gates before the second team joined the codebase. Org structure and system structure end up matching whether you plan for it or not — plan for it.

The org chart is the architecture

Melvin Conway's 1968 observation still holds: any system reflects the communication structure of the organization that built it. On a large-scale web project, this shows up fast — two teams that don't talk end up with two ways of validating input, two deploy pipelines, or two definitions of "done."

The practical move isn't to fight this. It's to decide the team boundaries first, then let the system boundaries follow — service ownership, repo layout, and on-call rotation should map to the same lines.

If you can't draw a diagram of who owns what before writing the first service, the codebase will draw it for you later — usually at the worst possible time, in a postmortem.

Monorepo vs. polyrepo is a real decision, not a preference

Google's own account of running one of the largest monorepos in the industry is worth reading before defaulting to "microservices, therefore polyrepo." Their engineering team documented the trade-off directly: a single repository gives you atomic cross-cutting changes, unified versioning, and one source of truth for what's actually deployed — at the cost of needing serious tooling investment (custom build systems, code search, ownership enforcement) to keep it navigable at scale.

Polyrepo trades that tooling cost for team autonomy and independent deploy cadence, at the cost of "who owns this shared library" becoming a standing organizational question instead of a one-time decision.

Dimension	Monorepo	Polyrepo
Cross-cutting refactors	Atomic, one commit	Coordinated multi-repo PRs, version bumps
Team autonomy	Lower — shared CI, shared conventions	Higher — each team picks its own pace
Tooling investment	High (custom build/search tooling at scale)	Lower per-repo, higher in aggregate (versioning, dependency graphs)
Failure blast radius	Broken build can block every team	Contained to the repo, until integration

Trunk-based development beats long-lived feature branches

The single practice most correlated with fast, reliable releases at scale is trunk-based development: short-lived branches, merged to a shared trunk at least daily, with feature flags gating anything not ready for users. This isn't a style preference — it's what avoids the multi-week merge conflict that eats a sprint.

Why long-lived branches actively hurt large teams

A feature branch that lives for three weeks accumulates drift from trunk every day it's open. The eventual merge isn't a formality — it's a second integration project, with its own bugs, on top of the feature you already built.

Feature flags solve the actual problem long-lived branches were trying to solve (hiding incomplete work) without the drift cost. Ship the code to trunk behind a flag; turn the flag on when it's ready.

CI/CD gates are the contract between environments

"We have a CI/CD pipeline" isn't a meaningful claim by itself. What matters is what each environment promotion actually gates on — and whether a human, a test suite, or nothing at all decides that a change is safe to promote.

A minimum viable gate structure

Dev → staging: automated tests pass, build succeeds, no manual gate.
Staging → production: integration tests pass against a production-like environment, plus a named human sign-off for anything touching payments, auth, or data migrations.
Production rollout: canary or percentage-based rollout with an automatic rollback trigger on error-rate or latency regression, not a manual "watch the dashboard" step.

Google's Site Reliability Engineering book devotes an entire chapter to this as "release engineering" — treating the release process itself as a piece of infrastructure with its own reliability requirements, not an afterthought bolted onto development.

Each promotion step gates on a specific, named condition — not a generic "review" step.

A pipeline with no gate is a very fast way to ship a very fast outage. The question isn't whether to gate — it's which gates are automated and which require a named person to say yes.

Feature flags accumulate their own debt

Feature flags solve the merge-conflict problem trunk-based development introduces, but they create a second-order problem if nobody owns cleanup: a flag left in the codebase six months after its feature fully launched is a permanent branch point nobody remembers the reason for. Every flag needs an owner and an expected removal date recorded when it's created, not discovered later during a "why is this still here" archaeology session.

Treat stale flag removal as its own small, regular task — a monthly sweep is enough on most teams — rather than a cleanup project that only happens when the flag count becomes embarrassing.

Code review at scale needs a written standard, not tribal knowledge

Google's engineering practices documentation lays out a standard most large teams reinvent badly: a reviewer's job is to approve a change once it's a clear improvement, not once it's perfect. Requiring perfection on every PR is how review queues back up and engineers start rubber-stamping.

What the standard actually asks reviewers to check

Does the change do what it claims, correctly?
Is it well-designed for the system it's joining — not just internally consistent?
Is it appropriately tested, with the tests actually exercising the new behavior?
Is it understandable to someone who didn't write it?

Writing this down and holding every reviewer to it — instead of "ask the senior engineer on the team" — is what lets a large team scale code review without either a bottleneck or a rubber stamp.

Risk management means naming the specific risks, not filling a template

A generic risk register with "scope creep" and "resource constraints" as line items doesn't change anyone's behavior. A risk register that names "the payments team's migration deadline slips past our checkout freeze window" gets acted on, because it's specific enough to own.

Name the risk in terms of a specific team, system, or date — not a category.
Assign one owner per risk, not a committee.
Review risks at the same cadence as the release calendar, not a separate meeting nobody attends.

Observability is a design decision, not a monitoring dashboard

Large-scale systems fail in combinations no single dashboard predicts — a slow database query that's fine alone becomes an outage when it coincides with a traffic spike and a cache miss. Google's SRE book frames observability around three signal types: metrics (aggregate numbers over time), logs (discrete events), and traces (a single request's path across services).

The mistake most large teams make with logging

Teams that bolt on logging after the fact end up with inconsistent formats across services — one team logs JSON, another logs plain text, a third logs nothing on the happy path and everything on error. Standardize a structured log format (with a request ID that threads through every service a request touches) before the second team starts writing code, not after the first incident where you needed it and didn't have it.

A trace that can't follow a single request across every service it touched is not observability — it's a collection of unrelated logs that happen to be searchable.

Postmortems turn incidents into institutional memory

An incident that gets fixed and forgotten will recur, usually on a different team that didn't know the first one happened. The SRE book's postmortem culture treats every significant incident as a source of process improvement, with one hard rule: postmortems are blameless.

What a useful postmortem actually contains

A precise timeline — when the problem started, when it was detected, when it was mitigated, when it was fully resolved. These are usually four different timestamps, not one.
The specific technical cause, not a generic category like "infrastructure issue."
Concrete follow-up actions with named owners and dates — not "we will improve monitoring," but "add a paging alert on queue depth exceeding a defined threshold, owned by the platform team, by a specific date."

A postmortem without owned, dated follow-ups is a story, not a process improvement. Track completion of postmortem action items the same way you'd track any other committed work — in the same backlog, visible to the same stakeholders.

Dependency management is a large-team problem, not a small one

A ten-person team can manually track its handful of shared libraries. A large-scale project with a dozen teams sharing internal packages needs an explicit versioning and deprecation policy, or "which version of the shared auth library is production actually running" becomes a question nobody can answer quickly during an incident.

This is a direct consequence of the monorepo/polyrepo decision above: monorepos make this easier by construction (one version, always), while polyrepos need an actual versioning discipline (semantic versioning, a deprecation window, and a way to know which consumers haven't upgraded) enforced as policy, not left to convention.

Technical debt needs to be visible, not just felt

On a large team, technical debt accumulated by one team becomes another team's incident eventually, and nobody connects the two unless the debt was tracked somewhere both teams can see. Treat technical debt as a backlog item with the same visibility as a feature request — sized, owned, and prioritized against new work, not a permanent line item nobody schedules.

Post-launch is not the end of the project

Large-scale web projects that treat launch as the finish line lose their best context exactly when they need it most — the team that built the system disperses before the operational runbook exists. Budget explicit time after launch for monitoring setup, on-call handoff, and a first-90-days incident review.

FAQ

Should we use Scrum or Kanban for a large-scale project?

Neither choice matters as much as whether the team actually holds to short iteration cycles and visible work-in-progress limits. Both frameworks fail the same way when skipped in practice.

How many environments do we actually need?

Three, at minimum: development, a production-like staging environment for integration testing, and production. Add a fourth (a shared QA environment) only if multiple teams need to demo independently before staging.

What's the single practice that matters most for a large team?

Trunk-based development with feature flags. It removes the merge-conflict tax that scales with team size and branch lifetime.

Do we need a monorepo to do trunk-based development?

No — trunk-based development works in polyrepo setups too. The monorepo/polyrepo choice is about code organization; trunk-based development is about branch lifetime and merge frequency.

Who should own the CI/CD pipeline itself?

A platform or DevOps function that serves all teams, not each feature team maintaining its own pipeline variant. Divergent pipelines are how "it works on my pipeline" becomes a real incident category.

How do we know our code review process is actually working?

Track review turnaround time and post-merge defect rate together. A fast turnaround with a rising defect rate means reviewers are rubber-stamping, not reviewing.

References

From the Destm engineering archive. For current work on this topic, start at Solutions or the blog.