External Issue Trackers Developer Guide

External Issue Trackers — Developer Guide

How a FreeITSM ticket gets linked to an issue in someone else's tracker, and why the machinery underneath is deliberately not about Jira. Shipped as #945–#947.

Jira is the first connector and the only one currently written, but almost nothing here is about Jira. This page is the contract; everything Jira-specific lives on Jira Connector — Developer Guide, which doubles as the worked example for anyone adding GitHub, GitLab or Azure DevOps.

⚠️ Status: engine only. There is no user interface yet — no settings screen, no button on a ticket, no workflow action. What exists is the layer all of that will sit on. The roadmap is in §9.

---

1. 📁 The files involved

Colour key: 🧠 shared rule · 🔌 provider · 🗄️ schema · 🧪 tests · 📄 docs

🎨	File	What it does
🧠	includes/integrations/IssueDoc.php	One description or comment, built once, rendered four ways (§4)
🧠	includes/integrations/IssueTrackerProvider.php	The abstract contract every tracker implements (§3)
🧠	includes/integrations/integrations.php	The service: load a connection, escalate, refresh status — and the company guard (§6, §7)
🔌	includes/integrations/JiraProvider.php	The first concrete connector — see its own page
🗄️	database/freeitsm.sql, includes/db_verify_schema.php, includes/db_verify_indexes.php	integration_connections + integration_links, their keys and indexes
🧪	tests/integrations/run.php	151 assertions, none needing a live tracker (§8)
📄	CHANGELOG.local.md, this wiki	logged as #945, #946, #947

Note what is not there: no new module folder, no cron yet, no changes to the tickets module at all.

---

2. The shape

Three layers. The middle one is new; the outer two already existed.

    WORKFLOW ENGINE  (exists — workflow/includes/engine.php)
    triggers · conditions · action registry · vars · dry run · step log
                        │  escalate_to_tracker  (not built yet)
                        │  ▲ tracker.* triggers (not built yet)
    ┌───────────────────┴──────────────────────────────────────┐
    │  INTEGRATION ENGINE  (this page)                         │
    │  link registry · IssueDoc · status categories ·          │
    │  company guard · escalate service · status refresh       │
    └───────────────────────┬──────────────────────────────────┘
                            │  IssueTrackerProvider (abstract)
        ┌───────────────────┼──────────────────┬────────────────┐
    JiraProvider      GitHubProvider     GitLabProvider   DevOpsProvider
      (built)            (later)             (later)          (later)

The bottom split is deliberately the same one as includes/messaging/MessagingProvider.php, which already puts Twilio and Meta Cloud behind a single contract while ingestion, ticket creation and the reply path stay ignorant of which is live. That pattern is proven in this codebase, so it was copied rather than reinvented.

The top connection is the important design decision. Escalation will be a workflow action, not code in the tickets module — because Workflows already has triggers, conditions, {{ticket.subject}} substitution, dry run and an audit trail. "When a ticket becomes type Bug, raise it in Jira" is then a rule a user writes, not a feature we build. The consequence is that V1 adds no hook to the tickets module whatsoever.

---

3. The contract — what is generic, what is yours

This is the table to read before writing a connector. Generic means the engine relies on it and you must comply. Provider-specific means it is only how one tracker happens to work, and copying Jira's shape into a different tracker would be a mistake.

Concept	Generic (the contract)	Provider-specific (illustrative)
Issue identity	every tracker has a stable id, and usually a human key	Jira's id is numeric, its key is OPS-412 and changes on project rename
Status	must map onto four categories: todo / in_progress / done / cancelled	Jira exposes statusCategory.key with three values and no cancelled
Body	build an IssueDoc, never a string	Jira Cloud wants ADF; Data Center wants wiki markup
Auth	credentials are a JSON blob, encrypted at rest	Jira Cloud is Basic email:token; Data Center is a Bearer PAT
Identity of us	testConnection() must return account_identity	Cloud gives accountId, Data Center gives a username
Target	opaque array passed to createIssue()	Jira needs project + issue type; GitHub needs only a repo
Optional features	declare via supports(); unsupported methods throw	Jira supports everything, GitHub has no issue types at all

What you must implement

Four abstract methods, because every tracker genuinely has them:

abstract public function createIssue(array $target, string $summary, IssueDoc $body, array $fields = []): array;
abstract public function fetchIssue(string $externalId): array;
abstract public function testConnection(): array;
abstract public function renderDoc(IssueDoc $doc);

Everything else — addComment(), addAttachment(), verifyWebhook(), pollChanges(), listProjects(), listIssueTypes() — has a default that throws "not supported for this provider". That is copied from MessagingProvider and it matters: if optional methods were abstract, the contract would collapse to whatever the weakest tracker can do.

supports() answers the same question for the UI, which needs to know before it renders a field. A thrown exception is the wrong way for a settings screen to discover that GitHub has no issue types.

fetchIssues() — the batch escape hatch

The default loops fetchIssue(), which is always correct. Providers that can read many issues in one call should override it, because the status-refresh cron is the hot path. One issue failing must not lose the rest.

⚠️ Never branch on a status name

Status names are per-project, per-workflow and renamed at will. Every decision in the system keys off status_category; status_name is stored for display only. This is the same lesson as tickets.merged_into_id — a merged ticket is identified by a column, never by a status called "Merged", because a user can rename it.

---

4. IssueDoc — one document, four renderers

The problem: every tracker wants a different body format.

Tracker	Format
Jira Cloud	ADF — Atlassian Document Format, a nested JSON document
Jira Data Center	wiki markup
GitHub / GitLab	Markdown
Azure DevOps	HTML

If core built strings, there would be four sets of string-building that drift. So core builds a document:

$doc = (new IssueDoc)
    ->heading('Raised in FreeITSM')
    ->para('Ticket ', IssueDoc::link($url, 'SD-1042'), ' · Jane Doe (Acme Ltd)')
    ->rule()
    ->para($ticket['description']);

…and asks for the shape it needs: toAdf(), toWikiMarkup(), toMarkdown(), toHtml(), toPlainText().

The renderers live in IssueDoc, not in the connectors. That is what makes a new tracker cheap — renderDoc() is usually one line returning an existing renderer, so adding GitHub means writing no renderer at all.

Deliberately small: heading, paragraph, link, bullet list, code block, rule. It is not a rich-text format. Anything bigger becomes a second thing that has to be kept safe against untrusted ticket text, and the house rule is that there is only ever one sanitiser (assets/js/safe-html.js) — so this stays small enough not to become a rival to it.

⚠️ Escaping is the renderer's job

Text is always treated as plain text and escaped for the target format. A requester writing "use the * wildcard" must not produce italics; <script> must not survive into an Azure DevOps description. Never pass markup in and expect it to survive.

⚠️ ADF is strict

ADF rejects empty text nodes and empty paragraphs, and a malformed document 400s the whole request with an unhelpful error. So the builders refuse to create empty blocks at all: ->para('') adds nothing, a leading ->rule() is swallowed, and an entirely empty document still renders as one valid paragraph rather than an invalid empty one.

---

5. 🗄️ The tables

integration_connections

One configured tracker instance — a Jira site, a GitHub org. credentials is a JSON blob encrypted at rest with the same helper as messaging_channels.credentials; webhook_secret likewise.

Two columns deserve a note:

• account_identity — who our token authenticates as, captured at connection test. Nothing reads it yet. It is half of echo suppression: once comments sync both ways, an inbound event authored by this identity is our own write coming back and must be dropped rather than re-imported. It is populated now because back-filling it for links that already exist is miserable.
• ingress_mode — webhook or poll. See §9.

integration_links

The spine: one row per "our work item ↔ their issue".

• entity_type is polymorphic from day one (ticket / problem / change) even though only ticket is ever written. It costs nothing now and would be a migration later.
• external_id is the provider's stable id, not the key — PROJ-412 becomes OPS-412 when a project is renamed, so the key is display only.
• UNIQUE (connection_id, external_id, entity_type, entity_id) — one issue may legitimately link to two tickets, never twice to the same one.

---

6. ⚠️ Multi-company: a connection is a connection

This is the part most likely to be got wrong, because tenant_id means three different things in three kinds of table. See Multi-Tenancy — Developer Guide §1 for the full table; the short version for this module:

integration_connections.tenant_id NULL = shared across every company (an MSP's own Jira). Set = pinned to that one company (a client with their own Jira).

That is the connection shape — the same as mailboxes and messaging channels, and the opposite of tickets, where NULL means "the Default company's".

Consequences:

• The admin list is deliberately unfiltered. Someone configuring routing needs to see every connection at once.
• Never scope it with activeTenantFilter(). That treats NULL as Default-owned and would hide every shared connection from every client company.
• A read that returns credentials is the exception to "capabilities guard writes, not reads". integrationsListConnections() returns has_credentials as a boolean and never the token. That is what makes an install-wide list defensible; without it, "deliberately unfiltered" would be a cross-company credential leak.

The guard

integrationsCompaniesCompatible(?int $entityTenantId, ?int $connectionTenantId, ?int $defaultTenantId): bool

Escalation is workflow-driven, and a workflow's conditions are editable by anyone who can author workflows. The wiki is blunt that inbound company routing must never become a workflow rule — it is a hardcoded synchronous membrane. This is the outbound twin of exactly that rule: without it, one mis-scoped workflow escalates Acme's ticket content into Globex's Jira.

So the check lives in code nobody can edit, and runs before any network call and before any write.

• Shared connection → accepts anything.
• Pinned connection → accepts only its own company.
• A NULL work item is resolved to the Default company before comparing, so a Default ticket can still reach a Default-pinned connection.
• No resolvable owner + a pinned target → refuse, never guess.
• On a single-company install every call is trivially true — correct rather than a bypass, because there is nothing to leak to.

---

7. The service

integrationsEscalate() is the one place an escalation happens. The workflow action and the manual button will both call it. There is deliberately no separate "automatic" implementation, because the guards would then exist twice and one copy would eventually drift.

Order of operations, which is itself the design:

1. Schema gate → 2. validate → 3. load connection → 4. company guard → 5. skip_if_linked → 6. dry-run returns here → 7. create the issue → 8. record the link.

Note that dry run returns before the network call. A workflow test that minted a real Jira issue would be an unacceptable surprise.

⚠️ The schema gate

Every read goes through integrationsSchemaReady(). An install that has not run Database Verification has no tables, and an unguarded query would throw inside the ticket view. "Not set up" must look like "no linked issues", never like an error — a missing gate of exactly this kind once produced an empty inbox.

Status refresh

integrationsRefreshConnection() batches every link on a connection into one provider call and writes only rows that actually moved, so a quiet day costs one read and no writes. An issue missing from the batch leaves its cached value alone rather than blanking it.

Only a change of category is reported as an event. A rename from "In Progress" to "In progress" is not something to wake a requester for.

---

8. 🧪 Testing without a live tracker

tests/integrations/run.php — 151 assertions, no database, no network:

php tests/integrations/run.php

The trick is that payload building and response parsing are separate protected methods rather than being inlined into the HTTP calls, so the suite subclasses a provider and stubs httpRequest() with a queue of canned [code, body] pairs. Everything that decides correctness — the ADF-vs-wiki choice, status mapping, error extraction, paging, auth headers — is therefore provable offline.

⚠️ Every negative assertion is paired with a positive control

"The asterisk did not become italics" proves nothing on its own; it is equally true of a renderer that dropped the text. So each escaping test also asserts the surrounding content survived.

This matters most for the company guard, which was verified in both directions by deliberate breakage:

Breakage	Result
guard made permissive (return true)	exactly the 5 refusal assertions fail
guard made refuse-everything (return false)	exactly the 10 permission assertions fail

A guard that refused everything would otherwise pass the whole negative half while being completely broken. Do the same for anything you add here.

That exercise also caught an assertion that fatalled rather than failing — hiding every test after it. Guard assertions that could receive the wrong type.

---

9. What is not built yet

Piece	State
Settings screen (/system/integrations/jira)	not built
Workflow action escalate_to_tracker + tracker.* triggers	not built
Ticket panel and the manual escalate button	not built
Poll cron	not built
Comment sync both ways, webhooks, echo suppression	V2
Field / priority / issue-type mapping, attachments, per-analyst tokens	V3
Issues raised in the tracker creating tickets	V4
GitHub, GitLab, Azure DevOps connectors	V5

Inbound will be webhook or poll, and both must produce the same canonical events — polling is a degraded webhook, not a second pipeline. A self-hosted install behind a firewall often cannot receive an inbound call, which is the same wall the WhatsApp channel hit; the difference is that trackers all expose a queryable REST API, so the firewalled case needs no relay, it just polls.

---

10. Adding a new tracker

The payoff, and the test of whether the engine is right:

1. Write includes/integrations/<Name>Provider.php extending IssueTrackerProvider.
2. Implement the four abstract methods. renderDoc() is usually one line.
3. Map the tracker's states onto the four categories.
4. Declare capabilities().
5. Add one line to the factory in integrations.php.

Adding GitHub should touch nothing else. If it needs a change to the link table, the service, the queue or the UI, the abstraction was wrong.

⚠️ Validate against Azure DevOps, not GitHub. GitHub is the easy one — everything will appear to fit and you will learn nothing. DevOps breaks the most assumptions: updates use JSON Patch documents rather than a fields object, states are per-process, targets are a tree of project + team + area path + iteration path, and the body is HTML. If the contract survives DevOps, the others are trivial.

---

11. 📄 Keeping this page honest

This page and the code must never disagree. It documents decisions, not just mechanics, and a decision that changed in code but not here is worse than no documentation — someone will trust it.

If you change any of the following, update this page in the same commit:

• the abstract method list, or which methods are optional
• the four status categories, or how a provider maps onto them
• IssueDoc's block types or any renderer's output shape
• either table's columns, or what tenant_id NULL means (§6)
• the company guard's behaviour in any case
• the order of operations in integrationsEscalate() (§7)

The design doc behind all of this is docs/design/external-issue-trackers.md in the app repo — local only, since docs/design/ is gitignored.

Jira-specific behaviour is on Jira Connector — Developer Guide, which has its own version of this note.