Two distinct upgrade failures: v2.4.0 → v2.6.2 (column relationTargetFieldMetadataId does not exist) AND v2.4.0 → v2.7.3 (universalIdentifier undefined in WorkspaceRolesPermissionsCacheService)

Summary

Attempted self-hosted upgrade from a clean v2.4.0 instance to the latest available image. Hit two distinct failures, both in workspace cache recomputation that runs at the start of the upgrade sequence — before the actual schema migrations apply:

1. v2.4.0 → v2.6.2: WorkspaceFlatViewFilterMapCacheService.computeForCache SELECTs relationTargetFieldMetadataId column that the corresponding schema migration is supposed to add later in the same sequence. The column doesn't exist yet → upgrade aborts at step 72 (2.5.0_NormalizeCompositeFieldDefaults).

2. v2.4.0 → v2.7.3 (skipping 2.6, from clean snapshot): WorkspaceRolesPermissionsCacheService.hasSettingsGatedObjectPermissions calls .find() on permission flags, gets undefined, then reads .universalIdentifier on it → TypeError. Same step (72), different code path.

Both failures happen before any DB writes, so DB stays clean (verified by Elestio snapshot restore between attempts).

The same workspace runs perfectly on v2.4.0. Pre-upgrade upgrade:status reports Up to date / 0 failures. Following the official upgrade procedure from the docs (and confirmed-safe by @prastoin in Discord).

Environment

• Initial server: twentycrm/twenty:v2.4.0 (Docker, official)
• Target image (attempt 1): twentycrm/twenty:v2.6.2
• Target image (attempt 2): twentycrm/twenty:v2.7.3
• Twenty SDK (custom app): twenty-sdk@2.4.0 + twenty-client-sdk@2.4.0
• Postgres: elestio/postgres:16
• Deployment: Elestio managed Docker Compose, single workspace
• OS: Ubuntu 7.0.0-15-generic
• ENABLE_DB_MIGRATIONS: "true" on server, "false" on worker

Workspace shape

• 1 workspace (bf6fe561-902d-43cf-a7f7-e65c69a895c4)
• 5 roles: Admin (system), Operátor (custom, created via API), Technik (custom, created via API), Member (default), My Twenty default function role (created by SDK defineRole for app logic-functions)
• ~5 custom objects, ~40 custom fields
• Multiple defineLogicFunctions (event-driven + cron), defineFrontComponent widgets, definePageLayout with FRONT_COMPONENT widgets
• Several saved views with view filters (uses viewFilter table)

Pre-flight (always clean on v2.4.0)

docker exec twenty-server yarn command:prod upgrade:status

APP_VERSION: v2.4.0

Instance
  Inferred version: 2.4.0
  Latest command:   AddApplicationIdToPublicDomain 1798000003000 (2.4.0) (instance fast)
  Status:           Up to date
  Executed by:      v2.4.0
  At:               2026-05-15T18:55:28.683Z

Workspace
  Servis HVAC (bf6fe561-902d-43cf-a7f7-e65c69a895c4)
    Inferred version: 2.4.0
    Latest command:   MigrateToBillingV2 1797000001000 (2.4.0) (workspace)
    Status:           Up to date
    Executed by:      v2.4.0
    At:               2026-05-15T18:55:28.694Z

Summary
  Instance: Up to date
  Workspaces: 1 up to date, 0 behind, 0 failed (1 total)

---

Failure #1: v2.4.0 → v2.6.2

Reproduction

1. Verify clean v2.4.0 state (as above).
2. Stop stack, bump image tag to twentycrm/twenty:v2.6.2 in compose, docker compose up -d.
3. Server boots; auto-runs upgrade sequence on startup (default behavior with ENABLE_DB_MIGRATIONS=true).

Expected

Server applies all migrations 2.4 → 2.6, workspace cache recomputes, app available.

Actual

Upgrade sequence enters workspace step 72 (2.5.0_NormalizeCompositeFieldDefaults), but workspace cache recomputation triggers a SELECT against viewFilter that includes the relationTargetFieldMetadataId column — which is added by step 76 (2.6.0_AddRelationTargetFieldMetadataIdToViewFilter). Workspace fails immediately, sequence aborts.

[Nest] LOG [UpgradeCommand] Initialized upgrade sequence: 77 step(s)
[upgrade] event=sequence.initialized stepCount=77 dryRun=false
[Nest] LOG [WorkspaceIteratorService] Running on workspace bf6fe561-902d-43cf-a7f7-e65c69a895c4 1/1
[Nest] LOG [WorkspaceCommandRunnerService] Upgrading workspace bf6fe561-902d-43cf-a7f7-e65c69a895c4 1/1
[upgrade] event=workspace.start workspaceId=bf6fe561-902d-43cf-a7f7-e65c69a895c4 index=1 total=1 dryRun=false

query failed: SELECT
  "ViewFilterEntity"."workspaceId",
  "ViewFilterEntity"."universalIdentifier",
  "ViewFilterEntity"."applicationId",
  "ViewFilterEntity"."id",
  "ViewFilterEntity"."fieldMetadataId",
  "ViewFilterEntity"."operand",
  "ViewFilterEntity"."value",
  "ViewFilterEntity"."viewFilterGroupId",
  "ViewFilterEntity"."positionInViewFilterGroup",
  "ViewFilterEntity"."subFieldName",
  "ViewFilterEntity"."relationTargetFieldMetadataId",   -- ⚠️ column added by step 76, sequence is at step 72
  "ViewFilterEntity"."viewId",
  "ViewFilterEntity"."createdAt",
  "ViewFilterEntity"."updatedAt",
  "ViewFilterEntity"."deletedAt"
FROM "core"."viewFilter" "ViewFilterEntity"
WHERE "ViewFilterEntity"."workspaceId" = $1

error: column ViewFilterEntity.relationTargetFieldMetadataId does not exist

[Nest] ERROR [WorkspaceIteratorService] Error in workspace bf6fe561-902d-43cf-a7f7-e65c69a895c4:
  column ViewFilterEntity.relationTargetFieldMetadataId does not exist

QueryFailedError: column ViewFilterEntity.relationTargetFieldMetadataId does not exist
    at PostgresQueryRunner.query (/app/node_modules/typeorm/driver/postgres/PostgresQueryRunner.js:219:19)
    at SelectQueryBuilder.loadRawResults (/app/node_modules/typeorm/query-builder/SelectQueryBuilder.js:2192:25)
    at SelectQueryBuilder.executeEntitiesAndRawResults (/app/node_modules/typeorm/query-builder/SelectQueryBuilder.js:2040:26)
    at SelectQueryBuilder.getRawAndEntities (/app/node_modules/typeorm/query-builder/SelectQueryBuilder.js:684:29)
    at SelectQueryBuilder.getMany (/app/node_modules/typeorm/query-builder/SelectQueryBuilder.js:750:25)
    at Promise.all (index 0)
    at WorkspaceFlatViewFilterMapCacheService.computeForCache (/app/packages/twenty-server/dist/engine/metadata-modules/flat-view-filter/services/workspace-flat-view-filter-map-cache.service.js:41:86)
    at /app/packages/twenty-server/dist/engine/workspace-cache/services/workspace-cache.service.js:220:26
    at Promise.all (index 3)

[Nest] ERROR [UpgradeSequenceRunnerService] Workspace steps ended with 1 failure(s). Aborting — cannot proceed to next instance step.
[upgrade] event=sequence.aborted failures=1 reason=workspace-failures

[Nest] LOG [UpgradeCommand] Upgrade summary: 0 workspace(s) succeeded, 1 workspace(s) failed
[upgrade] event=summary totalSuccesses=0 totalFailures=1 dryRun=false
[Nest] ERROR [UpgradeCommand] Upgrade failed: Upgrade completed with 1 workspace failure(s)
[upgrade] event=aborted

Post-failure state

After failure, upgrade:status shows DB in mid-state:

APP_VERSION: v2.6.2
Instance
  Inferred version: 2.5.0
  Latest command:   EncryptTotpSecrets 1798000009000 (2.5.0) (instance slow)
  Status:           Behind
  Executed by:      v2.6.2
Workspace
  Failed at: NormalizeCompositeFieldDefaults 1778000001000 (2.5.0) (workspace)
    Servis HVAC: Status: Failed, Executed by: v2.6.2, Error: column ViewFilterEntity.relationTargetFieldMetadataId does not exist

DB now has 2.5.0 instance-level changes applied but workspace upgrade is Failed. Rolling back image tag to v2.4.0 results in HTTP 502 — old server can't read mid-state DB. The failed marker also blocks retry under any image (server sees failed workspace and refuses to retry without a reset CLI helper).

Recovery: Elestio snapshot restore to pre-attempt state.

---

Failure #2: v2.4.0 → v2.7.3 (after snapshot restore to clean v2.4.0)

Reproduction

1. Restore Elestio snapshot of clean v2.4.0 → verify upgrade:status shows Up to date / 0 failures.

2. Stop stack, bump image tag to twentycrm/twenty:v2.7.3, docker compose up -d.

3. Server starts (HTTP 200) but UI shows skeleton loading state — /metadata returns empty.

4. Run dry-run to capture clean stack:

   docker exec twenty-server yarn command:prod upgrade --dry-run -v
   

Expected

Dry-run plans all 83 sequence steps and reports planned changes without errors. Subsequent real upgrade applies them.

Actual

Sequence enumerated successfully (83 steps including the new 2.6.0_AddRelationTargetFieldMetadataIdToViewFilter fast-instance step that was missing-in-order in v2.6.2). Failure happens during the per-workspace cache recomputation at step 72 — different stack from Failure #1:

[Nest] LOG [UpgradeCommand] Initialized upgrade sequence: 83 step(s)
... (steps 1..71 enumerated) ...
  [72] workspace — 2.5.0_NormalizeCompositeFieldDefaultsCommand_1778000001000 (2.5.0)
  [73] fast-instance — 2.6.0_RenamePermissionFlagToRolePermissionFlagFastInstanceCommand_1778235340020 (2.6.0)
  [74] fast-instance — 2.6.0_PermissionFlagSyncableEntityFastInstanceCommand_1778235340021 (2.6.0)
  [75] fast-instance — 2.6.0_LinkRolePermissionFlagToPermissionFlagFastInstanceCommand_1778235340022 (2.6.0)
  [76] fast-instance — 2.6.0_AddRelationTargetFieldMetadataIdToViewFilterFastInstanceCommand_1798000005000 (2.6.0)
  ... (steps 77..83) ...

[Nest] LOG [WorkspaceIteratorService] Dry run mode: No changes will be applied
[Nest] LOG [WorkspaceIteratorService] Running on workspace bf6fe561-902d-43cf-a7f7-e65c69a895c4 1/1
[Nest] LOG [UpgradeAwareRepositoryProxy] [upgrade-proxy] strip relation RolePermissionFlagEntity.permissionFlag -> PermissionFlagEntity
[Nest] ERROR [WorkspaceIteratorService] Error in workspace bf6fe561-902d-43cf-a7f7-e65c69a895c4:
  Cannot read properties of undefined (reading 'universalIdentifier')

TypeError: Cannot read properties of undefined (reading 'universalIdentifier')
    at /app/packages/twenty-server/dist/engine/metadata-modules/role/services/workspace-roles-permissions-cache.service.js:187:156
    at Array.find (<anonymous>)
    at WorkspaceRolesPermissionsCacheService.hasSettingsGatedObjectPermissions
      (/app/packages/twenty-server/dist/engine/metadata-modules/role/services/workspace-roles-permissions-cache.service.js:187:95)
    at WorkspaceRolesPermissionsCacheService.computeForCache
      (/app/packages/twenty-server/dist/engine/metadata-modules/role/services/workspace-roles-permissions-cache.service.js:132:69)
    at process.processTicksAndRejections (node:internal/process/task_queues:104:5)
    at async /app/packages/twenty-server/dist/engine/workspace-cache/services/workspace-cache.service.js:220:26
    at async Promise.all (index 5)
    at async WorkspaceCacheService.recomputeDataFromProvider (/app/packages/twenty-server/dist/engine/workspace-cache/services/workspace-cache.service.js:228:26)
    at async /app/packages/twenty-server/dist/engine/workspace-cache/services/workspace-cache.service.js:92:36
    at async /app/packages/twenty-server/dist/engine/twenty-orm/storage/promise-memoizer.storage.js:27:31

[Nest] ERROR [UpgradeSequenceRunnerService] Workspace steps ended with 1 failure(s). Aborting — cannot proceed to next instance step.
[upgrade] event=sequence.aborted failures=1 reason=workspace-failures

[Nest] LOG [UpgradeCommand] Upgrade summary: 0 workspace(s) succeeded, 1 workspace(s) failed
[upgrade] event=summary totalSuccesses=0 totalFailures=1 dryRun=true
[Nest] ERROR [UpgradeCommand] Upgrade failed: Upgrade completed with 1 workspace failure(s)

Note on the upgrade-proxy log line

The line [upgrade-proxy] strip relation RolePermissionFlagEntity.permissionFlag -> PermissionFlagEntity appears immediately before the error. This suggests the UpgradeAwareRepositoryProxy is stripping the permissionFlag relation from RolePermissionFlagEntity for cross-version compatibility — possibly leaving the .find() lookup with no target to match against, which then returns undefined.

Same error on real (non-dry-run) upgrade

Running command:prod upgrade (without --dry-run) produces the identical stack. Real upgrade also fails before applying any migration, so DB remains in v2.4.0 state (verified via upgrade:status after rollback).

---

Root cause hypotheses

#1 (v2.6 ordering bug)

The workspace cache recomputes during upgrade workspace steps before the cross-version schema migrations have applied. The cache SELECT uses the target entity schema (which references relationTargetFieldMetadataId), but the schema migration that adds the column runs after the per-workspace step in the sequence. Either:

• The workspace cache should be deferred until after all schema-adding migrations have applied, OR
• The UpgradeAwareRepositoryProxy should also strip columns that don't yet exist in the DB (it appears to handle relations like RolePermissionFlagEntity.permissionFlag per failure #2's log line, but not new columns on viewFilter).

#2 (v2.7 universalIdentifier null)

hasSettingsGatedObjectPermissions calls .find() on a permission-flag collection, gets undefined, then dereferences .universalIdentifier. The [upgrade-proxy] strip relation line just above suggests the proxy stripped the relation, leaving the .find() with nothing to match. Possibilities:

• One of our 5 roles (likely the SDK-created defineRole "default function role") has a RolePermissionFlag row pointing at a PermissionFlag.universalIdentifier that doesn't exist in the v2.7 catalog (renamed/removed in v2.6 RenamePermissionFlagToRolePermissionFlag migration).
• permissionFlag rows have universalIdentifier=null and the proxy doesn't normalize them.

Either way, the cache compute should probably be defensive (skip permission flag entries that can't be resolved instead of throwing), or the upgrade sequence should run Backfill migrations before recomputing cache.

---

What I've tried

• ✅ Verified clean DB state before each attempt (upgrade:status: Up to date / 0 failures on v2.4.0).
• ✅ Verified Elestio snapshot restore produces clean DB (no residual failed markers from v2.6.2 attempt).
• ✅ Attempt #2 also tested with --dry-run -v — same error, same stack, no DB writes.
• ✅ Verified workspace runs perfectly on v2.4.0 after rollback (HTTP 200, full UI, all 18 records visible in test view).
• ❌ Did not try deleting custom roles before upgrade (would lose Operátor/Technik role config; reproducible via setup scripts but disruptive).
• ❌ Did not try manually adding relationTargetFieldMetadataId column before upgrade (didn't want to corrupt schema further).

Questions for maintainers

1. Failure #1: Is the v2.6 ordering bug already tracked? Is the fix to defer workspace cache recomputation until after schema migrations, or to extend UpgradeAwareRepositoryProxy to strip unknown columns?
2. Failure #2: Is there a known issue with defineRole SDK roles + v2.6 RenamePermissionFlagToRolePermissionFlag migration? Should SDK-created roles be deleted/recreated, or is there a backfill we're missing?
3. Is there a CLI helper to dump/inspect the workspace's role + rolePermissionFlag + permissionFlag state pre-upgrade?
4. Is there a CLI helper to reset failed upgrade state after a failed attempt, short of snapshot restore? (Failure #1 left the DB stuck in Failed state — only snapshot rollback recovered.)
5. Would a sanitized DB dump of the v2.4.0 state help as a reproducer?

Current state: rolled back to v2.4.0, fully functional, snapshot of v2.4.0 state preserved for reproduction if needed.

---

cc @prastoin (per Discord conversation)