Windows Linux Subsystem dev setup
This is an opinionated guide for a science-oriented development setup on Windows 10.
- Windows Subsystem for Linux (WSL)
- Pipenv (Pragmatic Installation)
- Visual Studio Code
- Jupyter Lab
- Miniconda (optional)
- Docker (optional)
The Windows Linux subsystem allows for taking advantage of some of the open-source-based development ease of the Linux World while staying in the Windows world. This will give you a CLI for Ubuntu only, however. If you want to get the full graphical desktop experience, we recommend to install Ubuntu, or one of its derivatives (Xubunut, Mint, etc.) as a dual-boot-system.
To get your Linux-based setup in Windows 10, follow these steps:
-
Install the Windows Linux Subsystem (WSL) v2
- In case you are experiencing issues with this step, here is some additional troubleshooting advice: login with your admin account and check again for updates; make sure to execute the kernel update as admin; if the kernel update doesn't execute, de-install and reinstall it
- Install Ubuntu in WSL
- Start, update and upgrade Ubuntu:
sudo apt update
, 'sudo apt upgrade'
Pipenv allows you to separate and reproducibly document the Python environments you are working in for different projects.
-
Install Pipenv. (If no warning skip the next step)
-
Add ~/local/bin to PATH. Open
/.profile and add this line- export PATH =”/.local/bin:$PATH”. -
Run $ source ~/.profile for the changes to take effect.
-
It's recommended to create a new environment with pipenv for each project
-
If you want to work with different Python versions in your environments, install pyenv
Visual Studio Code is a highly customisable code and text editor that is able to deliver an IDE-like experience via a vast ecosystem of extensions for all popular programming languages.
- Install Visual Studio Code in Windows
- Install WSL extension for VSC
- Install Python extension for VSC
Package Installation and Environment Setup:
- Go to your project directory and install the packages as required.
- This will create a new environment for your project.
- Select this new environment in the drop-down list when prompted for python interpreter.
While VS Code has also extensions for Jupyter Notebooks (a browser-based platform for interactive programming in several languages, like Julia, Python, and R), Jupyter Lab is more powerful, and closer to what you will be most likely to share with colleagues and/or the public to make your research transparent.
Here is a short guide on best practices for getting and working with it:
Install Miniconda, an alternative to pipenv that is sometimes preferable, depending on your projects dependencies.
- Install Miniconda within Ubuntu
- run
conda config --set auto_activate_base false
so that conda does not automatically activate (and messes up environments that you might have created in pipenv or similar) - Create a conda env to get a python virtual environment
Docker allows to virtualise whole operating systems, which makes it suitable for the development of larger software projects that are meant to be reproducibly deployed in a server environment and encompass several pieces of software (e.g. data collection tools, databases, and data analysis tools) requiring complex, entangled setups.
- Install and setup Docker:
- Update to Windows 10 build 2004 or later
- Install docker desktop with WSL 2 backend
- Activate the WSL 2 backend
This Wiki is curated by the Social Media Observatory, which is hosted by the Leibniz Institute for Media Research | Hans-Bredow-Institut and supported by the Research Institute Social Cohesion. A pretty version can be found here.
- Instagram-Tools
- Twitter-Tools
- Wikipedia-Tools
- Facebook-Tools
- YouTube-Tools
- Telegram-Tools
- Smaller Platform Tools
- Cross-Platform Tools
- General News Scrapers
- Secure Storage and Archiving
- Data Anonymization Tools
- Data Publishing