Home
Welcome to the Proteomic-Data-Manager wiki! Proteomic Data Manager is a Web-based, flexible, open-source platform for automated high-throughput MS-based omics. This platform supports a variety of existing tools, allowing for a fully autonomous workflow from data collection to data backup, processing with different 3rd party software, and finally generating tables and figures for data visualization. The platform is built with Python Django, JavaScript, and HTML and works with Raw File Uploader and Processor.
The following diagram illustrates the key steps in the data management platform.
-
Mass spec data is automatically transferred from individual control PCs to the server through the Raw File Uploader. The upload process is vendor independent and can work with any MS system that uses files or folders for storage, as the Raw File Uploader performs the upload based on OS-level file changes, rather than vendor-specific scripts. For more information, see the Raw File Uploader.
-
Once the server receives the raw file, it creates a SampleRecord with all the metadata and converts the data using a Docker command (the default is https://github.com/phnmnl/container-pwiz). It also creates a cache file for display of the MS1 plot and saves the files to four storage devices based on user settings.
-
If a default data processing protocol is specified, a processing task will be automatically added to the Data Analysis Queue and processed by workers (processors) installed on data processing computers. The results will then be uploaded to the server once processing is complete.
-
Finally, users can view the processed data through visualization nodes or through the R or Python environment in Jupyter Notebook.
To ensure data redundancy, all raw and processed data is automatically backed up to four different storage devices as specified during installation.
The platform is installed as Docker containers, so you need to have Docker software installed first. The deployment process should take less than 5 minutes, see the how to install
The platform can be installed on any hardware that supports Docker, as we have tested it on a variety of systems including a Raspberry Pi 4 with 4GB of RAM, a desktop PC running Windows 10, an Ubuntu server running version 22.04, an ESXI virtual machine, and a cloud deployment.
This platform supports virtually any file or folder-based MS system. The Raw File Uploader performs uploads based on OS-level file changes, rather than vendor-specific scripts, making it compatible with any MS system that uses files or folders for storage. For more information, see the Raw File Uploader.
The conversion and extraction process use a 3rd party Docker container, with the default tool (https://github.com/phnmnl/container-pwiz) already supporting many platforms. Any other Docker image can be used, and the conversion and extraction process is not strictly necessary, providing more information and flexibility.
There are various methods for utilizing 3rd party applications, depending on the type of tool.
- Windows applications (have to support CMD mode), using processing Node and Processor
- Python packages, processing Node or Jupiter Notebook
- Docker containers, conversion_settings in usersettings or Jupiter Notebook
- Other 3rd party software, REST API
A demo site can be accessed at http://proteomicsdata.com:8090, there is no processing functionality provides here, default user is admin/proteomicsdatamanager
YouTube tutorial channel: https://www.youtube.com/channel/UCahxAV0zD4LCYiL71-UNeAA
User community site: https://community.proteomicsdata.com/