epic: Separate CLI and API processes

[gabrielle-ong]

Goal

• Cortex will primarily be an OpenAI API equivalent server
• Cortex CLI should be a layer on top of the API Server
• CLI and server are now two distinct binaries

Subtasks

• planning: Cortex CLI should be a layer on top of the API Server #1492
• epic: installer and uninstaller for cortex cli and cortex api binaries #1480

Discussion:

#1386

Success Criteria (windows, max, linux)

• Installer: Installer installs 2 different binaries
• Uninstaller: Uninstalls 2 different binaries
• Cortex update replaces the app, installer, uninstaller and binary file (without installing cortex.llamacpp)
• all CLI commands will start API server, if not running
• Edge case: User run cortex update on v1.0.0 → prompted to install cortex-server as with 1.0.1

[hiento09]

Hey guys @dan-homebrew @vansangpfiev @gabrielle-ong @namchuai @nguyenhoangthuan99 @0xSage , I’d like to present an idea for the installer and updater when we separate the CLI and server into two distinct binaries, as follows:

Given that:

In the current updater, the user runs the cortex update command, which causes the cortex process to download the new binary version and replace the existing binary. However, other components of the installer, such as the uninstaller and post-uninstall script, are not updated because we only replace the binary file, not the entire app version on the user's machine.

Solution:

Instead of the current cortex update only downloading and replacing the binary file, the cortex update command will pull the installer file and install it in headless mode using the following commands:

• macos: touch /var/tmp/cortex_installer_skip_postinstall && sudo installer -pkg cortexcpp-nightly-network.pkg -target / && rm /var/tmp/cortex_installer_skip_postinstall
• Linux: echo -e "n\n" | sudo SKIP_POSTINSTALL=true apt install -y --allow-downgrades ./cortexcpp-nightly-network.deb
• Windows:
  • Powershell Start-Process -FilePath ".\cortex-0.5.0-154-windows-amd64-network-installer.exe" -ArgumentList "/VERYSILENT /SUPPRESSMSGBOXES /NORESTART /SkipPostInstall" -Wait
  • CMD start /wait "" network-setup.exe /VERYSILENT /SUPPRESSMSGBOXES /NORESTART /SkipPostInstall
    In these commands, we skip the postinstall process, which installs the cortex.llamacpp engine. This allows the installer to replace only the app, installer, uninstaller, and the new binary file. With this approach, the Cortex code no longer needs to handle the complexity of replacing two binaries; it can simply run the commands above to update both binaries and the scripts in the installer and uninstaller simultaneously.

What do you guys think of this idea? Please leave your comments. Thanks!

[vansangpfiev]

Hey guys @dan-homebrew @vansangpfiev @gabrielle-ong @namchuai @nguyenhoangthuan99 @0xSage , I’d like to present an idea for the installer and updater when we separate the CLI and server into two distinct binaries, as follows:

Given that:

In the current updater, the user runs the cortex update command, which causes the cortex process to download the new binary version and replace the existing binary. However, other components of the installer, such as the uninstaller and post-uninstall script, are not updated because we only replace the binary file, not the entire app version on the user's machine.

Solution:

Instead of the current cortex update only downloading and replacing the binary file, the cortex update command will pull the installer file and install it in headless mode using the following commands:

• macos: SKIP_POSTINSTALL=true && sudo installer -pkg cortex-1.0.0-rc1-mac-universal-network-installer.pkg -target /
• Linux: echo -e "n\n" | SKIP_POSTINSTALL=true && sudo apt install -y --allow-downgrades ./cortex-1.0.0-165-linux-amd64-network-installer.deb
• Windows: Start-Process -FilePath ".\cortex-0.5.0-154-windows-amd64-network-installer.exe" -ArgumentList "/SkipPostInstall /VERYSILENT /SUPPRESSMSGBOXES /NORESTART" -Wait

In these commands, we skip the postinstall process, which installs the cortex.llamacpp engine. This allows the installer to replace only the app, installer, uninstaller, and the new binary file. With this approach, the Cortex code no longer needs to handle the complexity of replacing two binaries; it can simply run the commands above to update both binaries and the scripts in the installer and uninstaller simultaneously.

What do you guys think of this idea? Please leave your comments. Thanks!

I totally agree with the above solution. That way, we won't need to change much of the updater code in the long-term.
While calling an installer from cortex can be tricky, it is doable.

[freelerobot]

Largely sounds good. From the user's perspective, what is the experience like? - Do new windows/views pop up (if so, then no go) - Is there an increased update wait time?

…

On Tue, Oct 15, 2024 at 7:35 AM vansangpfiev ***@***.***> wrote: Hey guys @dan-homebrew <https://github.com/dan-homebrew> @vansangpfiev <https://github.com/vansangpfiev> @gabrielle-ong <https://github.com/gabrielle-ong> @namchuai <https://github.com/namchuai> @nguyenhoangthuan99 <https://github.com/nguyenhoangthuan99> @0xSage <https://github.com/0xSage> , I’d like to present an idea for the installer and updater when we separate the CLI and server into two distinct binaries, as follows: Given that: In the current updater, the user runs the cortex update command, which causes the cortex process to download the new binary version and replace the existing binary. However, other components of the installer, such as the uninstaller and post-uninstall script, are not updated because we only replace the binary file, not the entire app version on the user's machine. Solution: Instead of the current cortex update only downloading and replacing the binary file, the cortex update command will pull the installer file and install it in headless mode using the following commands: - macos: SKIP_POSTINSTALL=true && sudo installer -pkg cortex-1.0.0-rc1-mac-universal-network-installer.pkg -target / - Linux: echo -e "n\n" | SKIP_POSTINSTALL=true && sudo apt install -y --allow-downgrades ./cortex-1.0.0-165-linux-amd64-network-installer.deb - Windows: Start-Process -FilePath ".\cortex-0.5.0-154-windows-amd64-network-installer.exe" -ArgumentList "/SkipPostInstall /VERYSILENT /SUPPRESSMSGBOXES /NORESTART" -Wait In these commands, we skip the postinstall process, which installs the cortex.llamacpp engine. This allows the installer to replace only the app, installer, uninstaller, and the new binary file. With this approach, the Cortex code no longer needs to handle the complexity of replacing two binaries; it can simply run the commands above to update both binaries and the scripts in the installer and uninstaller simultaneously. What do you guys think of this idea? Please leave your comments. Thanks! I totally agree with the above solution. That way, we won't need to change much of the updater code in the long-term. While calling an installer from cortex can be tricky, it is doable. — Reply to this email directly, view it on GitHub <#1415 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AQVWFCCBRX42LWHW2W7JDYDZ3RIL7AVCNFSM6AAAAABPJ5KC3WVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIMJSGUZDCMZRGM> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[hiento09]

Largely sounds good. From the user's perspective, what is the experience like? - Do new windows/views pop up (if so, then no go) - Is there an increased update wait time?
…
On Tue, Oct 15, 2024 at 7:35 AM vansangpfiev @.> wrote: Hey guys @dan-homebrew https://github.com/dan-homebrew @vansangpfiev https://github.com/vansangpfiev @gabrielle-ong https://github.com/gabrielle-ong @namchuai https://github.com/namchuai @nguyenhoangthuan99 https://github.com/nguyenhoangthuan99 @0xSage https://github.com/0xSage , I’d like to present an idea for the installer and updater when we separate the CLI and server into two distinct binaries, as follows: Given that: In the current updater, the user runs the cortex update command, which causes the cortex process to download the new binary version and replace the existing binary. However, other components of the installer, such as the uninstaller and post-uninstall script, are not updated because we only replace the binary file, not the entire app version on the user's machine. Solution: Instead of the current cortex update only downloading and replacing the binary file, the cortex update command will pull the installer file and install it in headless mode using the following commands: - macos: SKIP_POSTINSTALL=true && sudo installer -pkg cortex-1.0.0-rc1-mac-universal-network-installer.pkg -target / - Linux: echo -e "n\n" | SKIP_POSTINSTALL=true && sudo apt install -y --allow-downgrades ./cortex-1.0.0-165-linux-amd64-network-installer.deb - Windows: Start-Process -FilePath ".\cortex-0.5.0-154-windows-amd64-network-installer.exe" -ArgumentList "/SkipPostInstall /VERYSILENT /SUPPRESSMSGBOXES /NORESTART" -Wait In these commands, we skip the postinstall process, which installs the cortex.llamacpp engine. This allows the installer to replace only the app, installer, uninstaller, and the new binary file. With this approach, the Cortex code no longer needs to handle the complexity of replacing two binaries; it can simply run the commands above to update both binaries and the scripts in the installer and uninstaller simultaneously. What do you guys think of this idea? Please leave your comments. Thanks! I totally agree with the above solution. That way, we won't need to change much of the updater code in the long-term. While calling an installer from cortex can be tricky, it is doable. — Reply to this email directly, view it on GitHub <#1415 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AQVWFCCBRX42LWHW2W7JDYDZ3RIL7AVCNFSM6AAAAABPJ5KC3WVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIMJSGUZDCMZRGM . You are receiving this because you were mentioned.Message ID: @.>

@0xSage

• Do new windows/views pop up (if so, then no go)? => No, all the commands above run the installer in headless mode. I tested them, and they work as expected.
• Is there an increased update wait time? => We use a network installation but disable the post-install script that pulls cortex.llamacpp from the internet. Instead, it only pulls the new binaries, which are less than 30MB. I tested this, and the update process is smooth and quick.

[freelerobot]

👍👍

…

On Tue, Oct 15, 2024 at 9:53 AM hiento09 ***@***.***> wrote: Largely sounds good. From the user's perspective, what is the experience like? - Do new windows/views pop up (if so, then no go) - Is there an increased update wait time? … <#m_2046471413798012038_> On Tue, Oct 15, 2024 at 7:35 AM vansangpfiev *@*.*> wrote: Hey guys @dan-homebrew <https://github.com/dan-homebrew> https://github.com/dan-homebrew <https://github.com/dan-homebrew> @vansangpfiev <https://github.com/vansangpfiev> https://github.com/vansangpfiev <https://github.com/vansangpfiev> @gabrielle-ong <https://github.com/gabrielle-ong> https://github.com/gabrielle-ong <https://github.com/gabrielle-ong> @namchuai <https://github.com/namchuai> https://github.com/namchuai <https://github.com/namchuai> @nguyenhoangthuan99 <https://github.com/nguyenhoangthuan99> https://github.com/nguyenhoangthuan99 <https://github.com/nguyenhoangthuan99> @0xSage <https://github.com/0xSage> https://github.com/0xSage <https://github.com/0xSage> , I’d like to present an idea for the installer and updater when we separate the CLI and server into two distinct binaries, as follows: Given that: In the current updater, the user runs the cortex update command, which causes the cortex process to download the new binary version and replace the existing binary. However, other components of the installer, such as the uninstaller and post-uninstall script, are not updated because we only replace the binary file, not the entire app version on the user's machine. Solution: Instead of the current cortex update only downloading and replacing the binary file, the cortex update command will pull the installer file and install it in headless mode using the following commands: - macos: SKIP_POSTINSTALL=true && sudo installer -pkg cortex-1.0.0-rc1-mac-universal-network-installer.pkg -target / - Linux: echo -e "n\n" | SKIP_POSTINSTALL=true && sudo apt install -y --allow-downgrades ./cortex-1.0.0-165-linux-amd64-network-installer.deb - Windows: Start-Process -FilePath ".\cortex-0.5.0-154-windows-amd64-network-installer.exe" -ArgumentList "/SkipPostInstall /VERYSILENT /SUPPRESSMSGBOXES /NORESTART" -Wait In these commands, we skip the postinstall process, which installs the cortex.llamacpp engine. This allows the installer to replace only the app, installer, uninstaller, and the new binary file. With this approach, the Cortex code no longer needs to handle the complexity of replacing two binaries; it can simply run the commands above to update both binaries and the scripts in the installer and uninstaller simultaneously. What do you guys think of this idea? Please leave your comments. Thanks! I totally agree with the above solution. That way, we won't need to change much of the updater code in the long-term. While calling an installer from cortex can be tricky, it is doable. — Reply to this email directly, view it on GitHub <#1415 (comment) <#1415 (comment)>>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AQVWFCCBRX42LWHW2W7JDYDZ3RIL7AVCNFSM6AAAAABPJ5KC3WVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIMJSGUZDCMZRGM <https://github.com/notifications/unsubscribe-auth/AQVWFCCBRX42LWHW2W7JDYDZ3RIL7AVCNFSM6AAAAABPJ5KC3WVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIMJSGUZDCMZRGM> . You are receiving this because you were mentioned.Message ID: @.*> @0xSage <https://github.com/0xSage> - *Do new windows/views pop up (if so, then no go)?* => No, all the commands above run the installer in headless mode. I tested them, and they work as expected. - *Is there an increased update wait time?* => We use a network installation but disable the post-install script that pulls cortex.llamacpp from the internet. Instead, it only pulls the new binaries, which are less than 30MB. I tested this, and the update process is smooth and quick. — Reply to this email directly, view it on GitHub <#1415 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AQVWFCCKSGV563F4TFYVOKDZ3RYRDAVCNFSM6AAAAABPJ5KC3WVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIMJSGY2TEOJSHE> . You are receiving this because you were mentioned.Message ID: ***@***.***>