‹ All essays

Architecture

From Kubernetes Helm Charts to ServiceNow Update Sets: Resolving Multi-Developer Conflicts Like a Platform Architect

The Scenario

You've got five senior developers parallel-building on a single DEV instance—two crafting ATF suites for incident workflows, two extending scoped HR app with business rules, one refactoring global Script Includes for Now Assist integration. UAT preview shows 47 collisions across 23 Update Sets; admin spends 2 hours manually merging while business screams about sprint delay. As the architect, you're asked: "What's the systematic approach experts use to make this never happen again?"

Key Questions Answered

Q: What's the root cause diagnosis approach?

A: Think database transaction isolation levels—your conflicts are classic lost update anomalies. Start with Update Set Preview on UAT (not DEV) to surface collisions: navigate to Retrieved Update Sets > select parent set > Preview. The collision tab shows exact record overlaps by sys_id. Cross-reference with Team Development Dashboard (sys_team_development.do) to map which dev instance owns each conflicting record. Run this query on sys_update_set table: State=In Progress^Parent is empty to find orphan sets. Root causes cluster into three buckets: parallel modifications to the same base record (Script Includes, Business Rules) — by far the most common — followed by scope mismatches (global vs scoped) and unmerged iterative fixes.

Q: What are the key concepts I need to master?

A: Map your Kubernetes concepts directly: Update Sets = Helm charts (packaging), Scoped Apps = Namespaces (isolation), Extensions = CRDs (safe augmentation). Extension vs Modification: Extending means adding new records (new Business Rules, new Script Includes) that layer on top of base behavior; modifying means editing the base record itself, which collides with everyone else editing it. Every captured change lands as a sys_update_xml row in the developer's update set — that is what preview compares. Scoped Apps sandbox changes (app scope isolation prevents global pollution). Update Set Strategy: Parent-Child hierarchy (one sprint parent pulls 50+ story children). ATF Testing: Automated Test Framework runs post-preview validation—your CI/CD equivalent for catching regressions before commit.

Q: What's the step-by-step solution pattern?

A: Architects use this 7-step production pattern to drive collisions down dramatically:

  1. Sprint 0 (Day -1): Admin creates GLOBAL parent Update Set: "Sprint-105-20260216". All story sets reference this.
  2. Dev Phase: Each dev creates story Update Set (e.g., STRY-101-HR-ScriptFix), unit tests in ATF, marks Complete.
  3. Push to UAT: Dev > Update Set > Push to Test (custom UI Action commits remotely). Preview immediately.
  4. Collision Triage: If preview fails, use Compare tool on conflicted records. Winner takes sys_id; loser merges diffs into winner's set.
  5. Pre-Prod Rollup: Devs set Parent= Sprint-105 on all story sets (sys_update_set.Parent field).
  6. UAT Final: Retrieve parent from DEV to PROD-like clone, preview/commit single set.
  7. Prod Commit: Admin retrieves parent once—5 minutes total.

Q: What are the common mistakes and anti-patterns?

A: A handful of anti-patterns cause most deployment pain:

  • Multiple unparented sets: dozens of orphan sets per sprint = admin nightmare. Fix: Always parent to sprint master.
  • Committing before preview: Blind commits overwrite prod. Fix: Mandatory preview + ATF pass.
  • Global scope creep: Scoped app changes leaking global. Fix: Lock global UI with ACLs during sprints.
  • No iterative merging: STRY101-v2 without pulling v1 changes. Fix: Retrieve parent before each iteration.

Q: How do I prevent this in future development?

A: Implement Team Development (your multi-branch GitLab): spin sub-dev instances from DEV clone, pull changes via dashboard. Enforce via Business Rule on sys_script_include insert/update: if record exists in active Update Set owned by different dev, throw warning banner. Add ATF suite to CI pipeline: post-commit, ATF runs 100+ tests validating no regressions. Scoped Apps + Extension Points only—no direct table mods.

Q: How do I measure success?

A: Track these production metrics weekly: Collision Rate (conflicts per preview <2%), Commit Time (admin effort <10 min/sprint), ATF Pass Rate (>98%), Hotfix Velocity (prod incidents from US <1/week).

The Translation Map

Your orchestration skills map directly to ServiceNow's deployment model:

Your WorldServiceNow EquivalentKey Difference
Kubernetes NamespaceScoped ApplicationServiceNow scopes enforce record isolation at sys_scope level
Helm ChartUpdate Set (Parent-Child)Hierarchical pulling; no versioning like chart repos
CRD/Operator ExtensionExtension (new records layered on base)Changes tracked as sys_update_xml rows; base record untouched
Git Branch ConflictPreview Collision TabVisual diff + sys_id resolution required
CI/CD PipelineATF + Preview WorkflowInstance-bound; no external runners native

Hands-On: See It Work

// Conflict Warning Business Rule on sys_script_include (before Insert/Update)
// Equivalent: Kubernetes admission webhook validating resource ownership
//
// Every change captured by an update set is stored as a sys_update_xml row
// named <table>_<sys_id>. So: check whether another IN-PROGRESS update set
// already contains an update for this Script Include, and warn the developer.
(function executeRule(current, previous) {
    var updateName = 'sys_script_include_' + current.getUniqueValue();

    var xml = new GlideRecord('sys_update_xml');
    xml.addQuery('name', updateName);
    xml.addQuery('update_set.state', 'In progress'); // states are strings, not numbers
    // Optionally exclude your own current update set:
    // xml.addQuery('update_set', '!=', gs.getPreference('sys_update_set'));
    xml.query();

    while (xml.next()) {
        var setName = xml.update_set.name.toString();
        var setOwner = xml.update_set.sys_created_by.toString();
        gs.addInfoMessage('Conflict warning: "' + current.getValue('name') +
            '" is already captured in update set "' + setName +
            '" (created by ' + setOwner + '). Coordinate before saving, ' +
            'or one of you will lose changes at preview time.');
    }
})(current, previous);

This Business Rule runs on every Script Include save and checks whether any other in-progress update set has already captured the same record. On overlap, it banners the update set name and creator so the two developers coordinate before preview time instead of after. It warns rather than blocks — hard-blocking saves on shared instances creates more problems than it solves. Deploy to global scope; the same pattern works for other tables like sys_script_client (use the matching sys_update_xml name prefix).

What Transfers vs What's Different

Transfers directly: Orchestration patterns (parent-child like Helm dependencies), conflict resolution (diff/merge like kubectl apply), CI validation (ATF=your pipelines). Your DBA transaction knowledge diagnoses sys_id clashes perfectly.

Requires new learning: Instance-cloning workflow (no Docker-like spins), sys_id immutability (unlike Kubernetes UIDs), scoped isolation enforcement. Trade-off: zero vendor lock via APIs, but instance-bound testing slows iteration vs cloud-native.

The Business Impact

Sprints ship on time, the admin's merge marathon shrinks to a single parent-set commit, and production hotfixes drop because collisions are caught in preview instead of after deployment. The avoided rollback and re-test cost is where the business case lives.

Common Traps to Avoid

  • Trap: Setting Parent too early—blocks UAT commits since parent=in progress. Solution: Set parents only on sprint-1 day.
  • Trap: Retrieving from UAT to PROD (loses parent hierarchy). Solution: Always retrieve from DEV source instance.
  • Trap: Modifying committed sets. Solution: New child set + mark original Ignore.
  • Trap: Ignoring preview collisions. Solution: Mandatory ATF post-preview; fail sprint if >2 collisions.
  • Trap: Global changes in scoped sprints. Solution: ACL lockdown + Extension Points only.

Your Next Steps

  • Deploy the conflict BR above to sys_script_include and sys_business_rule today—monitor via /sys_team_development.do.
  • Create sprint parent US now; have team parent all active story sets by EOD.
  • Build ATF suite with 20+ tests for your top 5 modified tables; integrate to Jenkins via REST API.
  • Clone DEV to new Team Dev sub-instance; assign 2 devs and measure collision drop next sprint.

This pattern scales from a five-person team to large multi-team programs across global and scoped apps.