Skip to content

Technical Design

Jump to bottom
Xyvran edited this page Aug 13, 2026 · 7 revisions

Model

Classes

Diagram

classDiagram
    class TGFSMetadata {
        +TGFSDirectory dir
    }
    class TGFSDirectory {
        +str name
        +TGFSDirectory parent
        +datetime created_at
        +datetime modified_at
    }
    class TGFSFileRef {
        +int message_id
        +str name
        +Dict mirrors
    }
    class TGFSFileDesc {
        +str name
        +str latest_version_id
        +datetime created_at
    }
    class TGFSFileVersion {
        +str id
        +datetime updated_at
        +int size
        +List message_ids
        +List part_sizes
        +Dict mirrors
    }

    TGFSMetadata "1" --> "1" TGFSDirectory : dir
    TGFSDirectory "1" --> "*" TGFSDirectory : children
    TGFSDirectory "1" --> "*" TGFSFileRef : files
    TGFSFileRef ..> TGFSFileDesc : message_id points to
    TGFSFileDesc "1" --> "*" TGFSFileVersion : versions
Loading

TGFSMetadata, TGFSDirectory and TGFSFileRef live in the metadata object. TGFSFileDesc and its versions are stored in their own channel message, which the TGFSFileRef points at.

TGFSMetadata (tgfs/core/model/metadata.py)

A global object to describe the folder structure and store the file indexes. The whole object content is serialized as JSON and stored as a pinned file in the Telegram channel.

Example:

{
  "type": "TGFSMetadata",
  "dir": {
    "type": "D",
    "name": "root",
    "createdAt": 1145141919810,
    "modifiedAt": 1145141919810,
    "children": [
      {
        "type": "D",
        "name": "a-directory",
        "createdAt": 1145141919810,
        "modifiedAt": 1145141919810,
        "children": [],
        "files": [
          {
            "type": "FR",
            "messageId": 114514,
            "name": "file-name.png",
            "mirrors": { "-1001234567890": 233 }
          }
        ]
      }
    ]
  }
}

The serialized shapes are defined as TypedDicts in tgfs/core/model/serialized.py.

TGFSDirectory (tgfs/core/model/directory.py)

The class to describe a folder. There can be sub-folders (children) and files. The files are described with TGFSFileRef. A directory also carries createdAt and modifiedAt timestamps (milliseconds since epoch); modifiedAt is bumped whenever a child directory or file is added, renamed or removed.

TGFSFileRef (tgfs/core/model/directory.py)

The class to describe the location of the actual TGFSFileDesc body and some essential file attributes.

The idea of this class is to minimize the size of the TGFSMetadata. Because TGFSMetadata will be updated frequently, and each update results in a de-serialization, serialization and re-upload, the cost can be very high if too much information is stored in it.

Another thought behind it is sometimes not all the file detail is needed. For example, when a user executes an ls operation towards a folder, the non-relevant file information (size, updated time) will not be retrieved. But when the user executes ls towards a file, the clients will find the actual file information body (TGFSFileDesc) using the messageId field in this object.

With channel redundancy enabled, mirrors maps a mirror channel id to the message id of the file descriptor copy in that channel. The field is omitted when redundancy is not in use, so metadata written before the feature still parses unchanged.

TGFSFileDesc (tgfs/core/model/file.py)

The class to describe file information. A file can have multiple versions (by updating a file), and the information of every version (TGFSFileVersion) is stored in this object. The file name is not serialized here — it comes from the TGFSFileRef that points at this descriptor.

Example:

{
  "type": "F",
  "versions": [
    {
      "type": "FV",
      "id": "d2d6bc6c-8d39-4efd-8028-2d4721ae4af3",
      "updatedAt": 1145141919810,
      "messageIds": [114513, 114515],
      "size": 1610612736,
      "mirrors": {
        "-1001234567890": [514, 515]
      }
    }
  ]
}

TGFSFileVersion (tgfs/core/model/file.py)

The class to describe the location of the actual file content. A file is split into parts of at most 1 GB each, so a version points at a list of messageIds (https://t.me/channelId/messageId is the link to each part) and carries the total size in bytes. Updating a file results in another file version (the original version will not be deleted unless the user does so with a special command).

Older metadata that predates multi-part uploads stores a single messageId instead of messageIds; it is still read and transparently converted into a one-element list.

With channel redundancy enabled, mirrors maps a mirror channel id to the message ids of the copies of each part in that channel, aligned with messageIds. A 0 entry means that part has no copy in that channel yet — for example when the mirror was added after the file was uploaded and the backfill has not reached it. Like on TGFSFileRef, the field is omitted when empty.

Clone this wiki locally