Skip to content

motain/codespaces-data-engineering

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

8 Commits
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Intro

The devcontainer from OneFootball data engineering. The first devcontainer in src installs software, dotfiles, and repos. The image is published in Github. The second devcontainer in .devcontainer runs the published image and it has the list of the repos used by the team.

Setup

# locally
gh cs create -R motain/codespaces-data-engineering -m basicLinux32gb
gh cs ssh
# inside the container
zellij attach --create <name of the session>
emacs
# check the creation of the devcontainer for problems
tail -f /workspaces/.codespaces/.persistedshare/creation.log
# Vscode development
gh cs code

Changelog

V2.1

  • Use public image and pre-build without hooks.
  • Use only fish.

V2.0

  • Use public image instead of pre-build.
  • Emacs client everywhere (before Zellij, git commit, and scrolledit).
  • CWD doesn't block Emacs.
  • Better binding for Zellij, similar to Chrome.

V1.1

  • Faster post-create container. Now doom emacs is first sync during the build time.
  • Zellij new bindings like tmux to avoid conflict with doom emacs. Use wezterm SendKey to remap to something more convinient.
  • Zellij more real estate by removing the status plugin at the bottom.
  • Better starship configuration.
  • Smaller atuin interface. The full screen version was losing the context.
  • Fix the umask madness of codespaces.
  • fish as a vterm shell.
  • Doom Emacs
    • Disable workspaces
    • User ranger + icons instead of dired
    • Enable lsp and tree-sitter
    • Enable OSC 52
    • Add python support
    • Isolated emacs per developer.
  • Fix cursor in fish. Now you have line for insert and block for command like vim and emacs
  • Workaround for the change directory in fish. See zellij-org/zellij#3184

V1.0

  • Initial import

Sharing your terminal with multiple cursor

Repeat all the steps above up to zellij to test the sharing your terminal feature

Development

The common operations are adding or updating dotfiles or adding new software. Add a new dotfile to src/.devcontainer/dotfiles and add a new entry to src/.devcontainer/Makefile. New software is installed in src/.devcontainer/local-features/of/install.sh. Every change requires to publish a new image and the sha of the image needs to be committed to .devcontainer/devcontainer.json

# local feature development
make build
make up

Tricks

The build takes more than 30 mins. To speed up the inner loop, we leverage docker caching and we temporary disable steps in the build itself src/.devcontainer/devcontainer.json

Publish

The token generated by Github for Codespaces cannot push an image to ghcr. You can create a personal token and login again.

make push

Secrets

The following secrets need to be created in https://github.com/settings/codespaces of the user creating the codespace.

ATUIN_LOGIN=atuin login -u USER -p PASSWORD -k PASSPHRASE
# permissions needed
# codespace, gist, read:org, repo, workflow, write:packages
GHP_TOKEN=ghp_xxx

Authors

Co-authored-by: Alberto Miorin <109069886+alberto-of@users.noreply.github.com>
Co-authored-by: Valery Lavrentiev <valery.lavrentiev@onefootball.com>
Co-authored-by: Marlon Rozindo <151554259+felixmarlon@users.noreply.github.com>