Text

· · to read

This post will fall into the “well duh…” category for a lot of people, but sometimes we need a reminder.

rsync in action

I got myself a new laptop recently (more on that another day), and wanted to back up filesfrom my old desktop to a portable drive, potentially for loading onto the new device. The main thing I wanted to back up was my photo library; most of the rest of my files are in the cloud/git repos, but I have a large “pre-cloud” library of family photos and consolidated backups from old computers that I’ve not managed to sort through and upload anywhere yet, so I wanted to make sure I had a copy of it. The library is hundreds of GB of various file types, from media to all sorts of metadata files used by the likes of iPhoto.

Try as I might, I could not get the library to transfer using the Windows 10 file explorer. Transfer speeds were abysmal (less than 20Kb/s for a USB-C drive capable of ~1Gb/s), and it would keep hanging on various files. When this happened, it would with a retry/skip action, so needed human interaction to keep going. At the speeds I was seeing it would’ve taken several days to copy over – and if the copy failed I had to start from scratch. Basically, Explorer it wasn’t a viable solution.

Last night, as I was dozing off to sleep, my brain reminded me that rsync exists to synchronise two directories, and WSL makes it easy to use on Windows. There are probably a billion Windows utilities that will do the same job, but I already have rsync in my WSL setup, so why complicate things further?

My source folder is E:\Pictures, and the destination drive is D:\. In WSL, this translates to /mnt/e/Pictures and /mnt/d/ respectively. There are a great many options in rsync, but I find -avz is a useful starting point for most jobs, and tend to only deviate from that if an initial run fails or I have a specific need. All in all the command I used to backup my photos was:

rsync -avz /mnt/e/Pictures /mnt/d/

Simplicity in itself. A few hours later – instead of days – and my library is safely backed up to the external drive and ready for its future home.

Text

· · to read

I ran into an issue last night where I couldn’t generate a new React application template using npx create-react-app my-app. Annoyingly, this was only broken in the WSL environment of my personal PC, where it had been a while since I’d had to use the command. On Windows, where I’d never run the command before, it worked fine. The error I received was:

You are running create-react-app 4.0.2, which is behind the latest release (5.0.0).
We no longer support global installation of Create React App.

This was odd, as I’d never globally installed create-react-app. Never the less, I followed the error’s suggested fix of running npm uninstall -g create-react-app. Unsurprisingly, it didn’t work. What followed was an hour of trying various other “fixes” from around the internet – update NPM/clear the NPM cache/update NPX… none of which worked for me.

In the end I resorted to fixing the issue through brute force – finding wherever this mystery instance of create-react-app was lurking and purging it from my system with good old rm. Using a combination of find and rm I found 2 places containing binaries. Removing these directly didn’t fix the problem either, but working my way further up the diectory tree to their common parent directory did.

As it turns out, I’d accidentally stumbled on the NPX cache, which is kept hidden away from the usual NPM cache. Mine was in a slightly odd place because I use NVM to manage my NodeJS versions, but you can find yours using npm config get cache, then looking for an _npx directory within the returned path. Delete the contents in there, and npx create-react-app my-app should start working again.

Or, to make it really easy, run npx clear-npx-cache.

Text

· · to read

Because Docker have changed their licensing and subscription TOS overnight, I’ve had to rebuild my development environment so it doesn’t use Docker Desktop on Windows anymore. What follows are notes I’ve made along the way on how I got this working on my particular laptop. This is not a tutorial! While the notes below might help you, they’re mostly a reminder to myself, in case I need to rebuild again, or adapt this into documentation for the rest of the team.

Docker on WSL

Uninstall Docker Desktop. Make sure WSL2 and a distro are installed and updated, then follow this guide by Jonathan Bowman.

Install docker-compose:

sudo apt-get install docker-compose

Git configuration in WSL

Setup Git to use your email address and username:

git config --global user.name "Your Name"
git config --global user.email "youremail@yourdomain.com"

Use the Windows Credential Manager, to share credentials across WSL/Windows:

git config --global credential.helper "/mnt/c/Program\ Files/Git/mingw64/libexec/git-core/git-credential-wincred.exe"

WSL Tweaks

Set the default WSL distro:

wsl -l --all #list your available distos
wsl --setdefault Ubuntu-20.04

Create the file %USERPROFILE%\.wslconfig, and add the following:

[wsl2]
memory=8GB              # How much memory to assign to the WSL2 VM.
processors=4        # How many processors to assign to the WSL2 VM.

Adjust the values as necessary. This limits the VMMEM process to a sensible amount of resource usage – it’ll consume everything it can otherwise.

Restart WSL:

wsl --shutdown

(Open a new WSL terminal to start it again)

Windows Terminal Tweaks

To create a command which opens a single tab which is split into four panes, with each pane set to a different service directory, open the settings.json file, and add the following in the commands array:

{
            "command":
            {
                "action": "wt",
                "commandline": "new-tab -p \"Ubuntu-20.04\" -d \"//wsl$/Ubuntu-20.04/home/chris/dev/graphql\" ; split-pane -V  -p \"Ubuntu-20.04\" -d \"//wsl$/Ubuntu-20.04/home/chris/dev/inbound-service\"; split-pane -H  -p \"Ubuntu-20.04\" -d \"//wsl$/Ubuntu-20.04/home/chris/dev/outbound-service\"; mf left; split-pane -H  -p \"Ubuntu-20.04\" -d \"//wsl$/Ubuntu-20.04/home/chris/dev/front-end\""
            },
            "name": "startdev"
}

The network-style formatting of the starting directory is necessary, otherwise the path won’t be parsed properly.

Other

Install Node, etc on WSL by following this guide in the Microsoft docs.

To get VS Code Remote working, open a WSL terminal, then run the following:

code .

This should automatically install the Remote server. Once done, a Remote editor can be opened in Code on the Windows side. Extensions for Remote are managed separately, so install any that are needed, such as Docker, etc.

To have the VS Code Remote window make sure Docker is started when it’s opened up, add the following to ~/.vscode-server/server-env-setup on the WSL side:

DOCKER_DISTRO="Ubuntu-20.04"
DOCKER_DIR=/mnt/wsl/shared-docker
DOCKER_SOCK="$DOCKER_DIR/docker.sock"
export DOCKER_HOST="unix://$DOCKER_SOCK"
if [ ! -S "$DOCKER_SOCK" ]; then
    mkdir -pm o=,ug=rwx "$DOCKER_DIR"
    chgrp docker "$DOCKER_DIR"
    /mnt/c/Windows/System32/wsl.exe -d $DOCKER_DISTRO sh -c "nohup sudo -b dockerd < /dev/null > $DOCKER_DIR/dockerd.log 2>&1"
fi