Skip to content

File Storage Locations Design

Ed Mozley edited this page Nov 21, 2026 · 3 revisions

📦 Choosing where FreeITSM keeps its files - design

Raised by @vsc55 on PR #350 · Status: step 1 under way - progress in File stores - Developer Guide (Ed, 21 Nov 2026) · Related: Storage usage (5.6.0), Files storage

In one paragraph

Today every module keeps its files in a fixed folder inside the application. This design lets an administrator choose a different folder for each kind of file - ticket attachments, documents, LMS content and so on - typically outside the web root, for example D:\Data\Ticket attachments. You move one store at a time: FreeITSM tests the new folder, you copy the files across, FreeITSM checks every file is there, then switches. You test, take a breather, and do the next one when you are ready. Nothing moves until you ask, and the old folder stays as your backup.

The idea, and why

@vsc55 noticed that the files FreeITSM stores live in about ten folders spread through the application, and each new one has to be remembered in a Docker volume, a web-server guard, the storage list and two debug tools - forms/images and label-fonts had already been missed once. He proposed one data folder outside the web root, one helper that every piece of code asks for a path, and a careful, visible move.

Ed's view (16 Nov 2026): "a scary move and would need to be incredibly careful - hold off". On 21 Nov 2026 he picked it up again with one change: a separate location per store, so a move can be done one module at a time.

How it works today (checked in the code, 21 Nov 2026)

Every store except Files has its folder fixed in the code, worked out from the code's own position (dirname(__DIR__) . '/tickets/attachments'), and that is repeated in each file that needs it - 21 files for ticket attachments alone. Files is the exception: its location is already a setting (files_storage_root) with a safe move.

The good news is that most stores record their files in the database relative to their folder, so the folder can change without touching a single row.

Store Folder today What the database holds Layout inside How the browser gets a file Files that build the path
Ticket attachments tickets/attachments email_attachments.file_path, relative - 12/12345/<hex>.pdf {id/1000}/{email id}/ PHP (get_attachment.php, portal, chat media) 21
Change attachments change-management/attachments change_attachments.file_path, relative - {change id}/<hex>.pdf {change id}/ PHP (get_attachment.php, REST) 5
Documents uploads/documents documents.storage_key, a file name flat PHP (documents/download.php and portal) ~8
Files uploads/files (already a setting) files_versions.storage_path, relative - yyyy/mm/dd/<hex>.bin dated PHP (files/download.php) 1 setting
Knowledge pictures uploads/knowledge knowledge_images.stored_name, a file name flat PHP (knowledge/image.php) 2
Emailed reports, API log archive, asset imports uploads/report-downloads, uploads/api-log-archive, uploads/asset-imports a file name, or none flat PHP, or never downloaded 1 each
Screen recordings recordings ticket_recordings.file_path with the folder name in it - recordings/123/<hex>.webm pending/, then {ticket id}/ PHP (get_recording.php) 4
LMS content lms/content lms_courses.launch_url, relative to the course's folder {course id}/ + the package's own files ⚠️ straight from the folder (iframe) 4
RFP uploads contracts/rfp-builder/uploads rfp_documents.file_path - ⚠️ a full path (C:\wamp64\www\...) flat never downloaded 4
Branding system/uploads/branding settings and tenants.logo_path with the folder name in it flat + portal/, companies/ ⚠️ straight from the folder (<img src>, CSS) ~13 + 1 JS
War room war-room/attachments warroom_attachments.stored_name, a file name flat PHP 1 (a constant)
Form pictures forms/images {form id}/<hex>.png, inside a form field's settings {form id}/ PHP (forms/image.php) 3
Label fonts system/uploads/label-fonts asset_label_fonts.stored_name, a file name flat PHP (label_font.php) 2

Four lists of these folders are kept in step by hand: storagePersistenceDirectories(), the Storage usage inventory, UPLOAD_HTACCESS_ROOTS and .gitignore - plus the Docker volumes, the nginx config and debug tool D009.

What has to be put right before a store can move

  1. RFP uploads store a full path in the database. It already breaks if the application folder moves, or between Windows and Docker. It must become relative (a one-off change in Database Verification) before anything else.
  2. Recordings and branding store the folder name with each file. Either strip it when reading, or rewrite the rows once - both are safe, and the rows keep working for an install that never moves anything.
  3. LMS content and branding are fetched by the browser straight from the folder. A folder outside the web root cannot be reached that way. Decided: they stay in the application folder for now and are revisited in a later release (serving them through PHP first).
  4. The war room clears its own folder: any file in it without a database row is deleted. Every store must therefore live in its own folder, never one shared with anything else - the screen enforces this.
  5. Small gaps found on the way: uploads/knowledge is missing from the Storage usage list, D009 does not probe uploads/report-downloads or uploads/asset-imports, LMS content has no web.config for IIS, and recordings and LMS course folders are never deleted when their ticket or course is.

The design

Three steps, each useful on its own and each safe to stop after.

Step 1 - one helper per store (nothing moves)

Every piece of code that saves, reads or deletes a file asks one function where that store lives, instead of building the path itself. In this step the function returns exactly today's folders, so nothing changes on disk, in the database or on screen. It is invisible to users - and it is what makes step 2 safe, because a setting read in 20 places but missed in the 21st would save a file in one place and look for it in another.

  • FreeITSM's own store layer, built at its own pace. PR #361 (@cloudcogsio) has a similar layer for putting stores in S3; Ed decided (21 Nov 2026) not to use it, so the migration moves at a pace set here, one store at a time. The helper is one registry of stores - an id, today's folder, and the setting that can override it - with one function to get a store's folder, and it is converted store by store, each conversion its own small change with its own tests.
  • The same step fixes the five problems above (relative RFP paths, the folder-name prefixes, the gaps), and replaces the four hand-kept folder lists with the one registry.
  • A test fails if any file under api/, includes/ or a module builds a store path by hand, so a new feature cannot quietly add a 22nd.

Step 2 - a folder you choose, per store

Each store gets a location on System → Storage:

  • In the application folder - today's, and the default. Nothing changes for anyone who never opens this screen.
  • A folder on this server - one you choose, per store.

The screen recommends a structure - one parent folder with a subfolder per module, outside the web root:

D:\Data\
    Ticket attachments\
    Change attachments\
    Documents\
    Knowledge pictures\
    Files\
    Form pictures\
    ...

Inside each folder, the layout stays exactly as it is today (12\12345\... for ticket attachments, 2026\11\21\... for Files). A move copies the contents as they are, so no database row has to change. Reorganising every store by date would mean rewriting every row, for no gain.

Keeping a path from the database safe

A folder path read from the database needs care - it is exactly what a copied database would carry somewhere it does not belong. So:

  • Only administrators with the Storage permission can set it, and every change is recorded (who, when, from where to where).
  • It is checked before it is accepted: an absolute path, no .., the folder exists and FreeITSM can write, read back and delete a test file there; it is not inside another store's folder or the application's code; on Docker, it is on a volume (a folder that is not would be wiped by the next rebuild). A folder the web server would serve to the public is refused for stores fetched through PHP, which is all of them once step 1 is done.
  • A copied database cannot write into the wrong install's files. When a folder is chosen, FreeITSM leaves a small marker file in it naming this installation and the store. If the database is ever copied - to a test install, a restored backup, a new server - and points at a folder whose marker names a different installation or store, FreeITSM refuses to write there and says why on System → Storage, rather than mixing two installs' files.
  • Storage usage shows the chosen folder for every store, so an unexpected location is easy to spot.

Step 3 - moving one store: the steps

On System → Storage, a store's Move walks through:

  1. Choose the folder. The recommended path is filled in; change it if you like.
  2. Test. FreeITSM writes, reads back and deletes a test file, and runs the checks above. Nothing is moved and nothing is switched.
  3. Copy the files - you do this. Copy the contents of the old folder into the new one, exactly as they are (Explorer, robocopy, rsync). The screen shows the exact source and destination to copy. Copy, do not move: the old folder stays as it is. FreeITSM does not copy files itself (Ed, 21 Nov 2026) - its job is to verify.
  4. Check. FreeITSM checks every file the database knows about for that store - not a sample - is in the new folder at the right size, and reports "4,812 of 4,812 found", listing any that are missing. Switching is not offered until every file is accounted for (a file already missing from the old folder is listed separately, so a gap that was there before the move cannot block it).
  5. Switch. From now on new files go to the new folder. Files that arrived while you were copying (mail is fetched every minute) are not lost: until you press Finished, a file not found in the new folder is read from the old one instead.
  6. Take a breather. Use FreeITSM as normal. Go back returns the store to the old folder in one click, because the old folder is still there.
  7. Finished. When you are happy, press Finished. FreeITSM checks again first and lists any file still only in the old folder - the ones that arrived during the copy - for you to copy across; Finished is only offered once there are none. From then on it stops looking in the old folder. FreeITSM never deletes the old folder for you; you delete it when you are ready.

Then the next store, whenever you choose.

Suggested order

Easiest and lowest-risk first, so the process is proved on something small:

  1. Form pictures, label fonts, war room
  2. Change attachments, documents, knowledge pictures, emailed reports, the API log archive and asset imports
  3. Screen recordings (after its folder-name fix)
  4. Ticket attachments - the biggest, and written to every minute by mail
  5. RFP uploads (after its full-path fix)
  6. Files keeps its own location setting on Files → Settings → Storage, shown on System → Storage as a row for completeness.

Docker

With one volume per store today, moving a store to a different folder inside the container needs that folder on a volume - the Test refuses one that is not. The simplest Docker setup becomes one volume for the recommended parent folder (/app/data) with every store chosen inside it, which is what @vsc55 originally proposed; installs that keep today's folders keep today's volumes.

What does not change

  • Any install that never opens the screen: same folders, same volumes, same everything.
  • The layout inside each store's folder, and the database rows (apart from the one-off fixes in step 1).
  • The web-server guards on today's folders stay, for every store still using them.
  • Branding and LMS content stay in the application folder in this design (revisited in a later release).

Not in this design

  • Branding and LMS content stay in the application folder; serving them through PHP so they can move is for a later release.
  • S3 for these stores (PR #361) is not part of it.
  • FreeITSM copying files itself - the administrator copies, FreeITSM verifies.

Decided

Date Decision
16 Nov 2026 Hold off: "a scary move and would need to be incredibly careful" (Ed).
21 Nov 2026 Go ahead with a design. A separate location per store, so stores can be moved one module at a time: update one, test, take a breather (Ed).
21 Nov 2026 The System screen recommends a structure (a parent data folder with a subfolder per store); you choose each store's folder yourself rather than one root that moves everything (Ed).
21 Nov 2026 Each move: FreeITSM tests it can write and read the folder, the files are copied, FreeITSM checks the files are there, then switches (Ed; refined here to check every file and to catch files that arrive during the copy).
21 Nov 2026 The location is found by one helper in the code; the administrator's choice is a setting it reads, empty meaning today's folder - never a path worked out once and saved, which a copied database would carry to the wrong install.
21 Nov 2026 Not built on PR #361 - Ed likes the idea but wants the migration handled at FreeITSM's own pace, so step 1 is FreeITSM's own store layer, converted one store at a time (Ed).
21 Nov 2026 Branding and LMS content stay where they are for now; revisit in another release (Ed).
21 Nov 2026 Folders organised by module: today's single uploads/ folder holds several modules' files, so each becomes its own store and folder - Documents, Knowledge pictures, Emailed reports, API log archive, Asset imports. Documents stays one folder, because the same documents are attached across tickets, notes, assets, articles and projects (Ed; Documents noted here).
21 Nov 2026 The administrator copies the files; FreeITSM verifies - no Copy for me (Ed).

FreeITSM

Getting Started

Modules

Multi-tenancy (planned)

Responses to feedback

Blue sky thinking

Bugs resolved

Pull requests

Links

Clone this wiki locally