Maciek Janicki’s website

Following science journals via RSS

2020-11-08T00:00:00+02:00

Working in academia means having to follow the current state of the research. The “usual” way to do it is downloading PDFs from websites of conferences and journals. It is needless to say that this is terribly inefficient: it requires us to remember when something new is expected to appear, manually navigating through the website etc. Even worse, recently there is a trend to increasingly rely on social media (Twitter, ResearchGate) and bloated, proprietary, data-mining “bibliography managers” (like Mendeley) to get suggestions on what to read. In this post, I describe an efficient, distraction-free and privacy-protecting workflow for keeping up with the latest research.

The general idea is to use RSS feeds to track updates of the journals in a text mode RSS reader. Furthermore, we want the link in the RSS item to send us directly to the PDF version of the paper (opening in our favourite reader), which usually requires a slight hack.

Use RSS to follow updates

Once upon a time, there was a universal way of following updates on any website: RSS feeds. The standard is dead-simple: each site exposes an XML file containing a list of recent “posts”. They typically contain a title, publication date, a short summary and a link to the full article. There are dozens of clients, both text-based and graphical, that you can use to subscribe to such feeds and gather all updates in one place. Without logging in to any accounts and giving away the information about which sources you follow and which articles you read.

Here’s how I browse the journal Journal for Data Mining & Digital Humanities (JDMDH) in my text-based client newsboat:

In order to get the list of articles as an RSS feeds, all you need to do is add the feed URL (e.g. https://jdmdh.episciences.org/rss/papers for that journal) to the newsboat’s list of URLs. Most sites will have such feed, but occasionally it might be hard to find (look for “RSS” links).

Shorten the way from an RSS item to PDF

In an ideal world, I would get the PDF on my desktop by hitting just one keystroke in newsboat. This is sometimes possible. Where it is not, a terminal-based web browser can minimize the pain and distraction caused by having to pass through a website.

In practice, most RSS feeds contain a link that leads not straight to the PDF, but to a website from which a PDF can be downloaded. In the easiest case, the URL straight to the PDF can be deduced from website URL. For example, the “Journal of Data Mining & Digital Humanities” refers to papers using an URL like https://jdmdh.episciences.org/3905, while the PDF can be accessed under https://jdmdh.episciences.org/3905/pdf. Thus, we always need to add a trailing /pdf to the link.

In order to implement this, let’s start by setting the web browser used by newsboat (i.e. the program called when you hit the keybinding for “open”) to a custom script. Edit the file ~/.newsboat/config and add the line:

browser ~/.newsboat/browser.sh

Then create the file ~/.newsboat/browser.sh with the following content:

case "$1" in
http://jdmdh.episciences.org/*)
    curl "$1"/pdf 2> /dev/null | zathura - &
    ;;
*)
    exec w3m "$1"
esac

Right now we have one rule for the “Journal of Data Mining & Digital Humanities” and a default rule at the end, which opens everything else using w3m. The rule for JDMDH executes the following command:

curl "$1"/pdf 2> /dev/null | zathura - &

which:

appends /pdf to the URL referenced in the RSS item,
downloads the PDF file and prints it to standard input,
launches zathura that reads the PDF from stdin (option -).

Now pressing o on an item in newsboat should pull up the PDF in zathura.

A little annoyance of this method is that you need to implement a separate rule for almost every source, because each of them might have a different way of changing the webpage URL to the one of the PDF - you need to figure out the rule by comparing those every time. Here’s one for arXiv:

http://arxiv.org/abs/*)
    PDFURL=$(echo "$1" | sed 's|^http://|https://|; s|/abs/|/pdf/|; s|$|.pdf|')
    curl "$PDFURL" 2>/dev/null | zathura - &
    ;;

Setup w3m to directly open PDFs

For sime sites, the relation between the URLs might be unpredictable or downloading with curl/wget might not work for some reason. Our default rule opens the site in w3m then. If there’s a “Download PDF” link somewhere, you can setup w3m to open the file directly in zathura by adding the following line:

application/pdf; zathura %s

to either ~/.mailcap or ~/.w3m/mailcap. Then all you need to do is to navigate the site to find the PDF link - if you open it in w3m, you can at least avoid looking at the ads and flashy webdesign.

Conclusion

The above workflow gives me a standard and minimum-distraction way to follow the newly appearing articles. Especially a look at arXiv, which has almost daily updates, allows me to keep up with the field. All sources are accessible with the same text-based interface, thanks to which I can better focus on the text. As in the other solutions that I present, it allows me to flexibly and seamlessly combine the most suitable applications for the job: newsboat for RSS, w3m for quick Web browsing without really looking at the site, zathura for reading PDFs. The fresh research is delivered straight to my desk.

Executing code chunks from Vim

2020-09-26T00:00:00+03:00

Recently “notebook” applications like Jupyter and RStudio have become a popular way of teaching and practicing programming. Their advantage is making programming interactive: you write several lines of code, then execute it and look what happens. You can tinker with your complex data structures and try unfamiliar functions out without the need to rerun the whole script each time. However, this also comes with the massive disadvantage: you are locked down to a GUI application and cannot edit the code with a decent text editor. In this tutorial, I show how to achieve the same effect using Vim, the standard REPL console, and tmux to connect them.

Note: tmux or “terminal multiplexer” is a program that allows you to manage multiple terminal emulators inside one text terminal or window. In the tutorial below, I assume that you are familiar with the basic concepts of tmux, like sessions, windows and panes. If not, I strongly recommend to check out this amazing piece of software.

Before we get started, here’s a short demo of what we’re aiming at:

You can see that the first time I hit Ctrl+E inside Vim, a new tmux window pops up, showing a session named repl-R and running the R interpreter. Then I can select chunks of code and press Ctrl+E again, which sends the selected code to the interpreter. Inside a tiling window manager (mine is dwm), it looks almost like an RStudio session, except that I’m using my favourite terminal-based tools. Furthermore, it be done exactly the same way for different text editors (e.g. Emacs or vis) and different interpreters (Python, Haskell, what have you). Let’s get down to implementing it.

Step 1. Sending keys to a tmux pane from outside

In the first step, we’re going to learn how to control an application running in a tmux pane by running shell commands from outside that pane.

Execute the following steps:

Start a new tmux session called “test” by running
```
tmux new -s test
```
in a terminal emulator. This will create the session and a new window with ID 0 inside it.
Open a completely new terminal window (even without running tmux inside, doesn’t matter) and execute the following command:
```
tmux send-keys -t test:0 ls Enter
```
Go back to the window running the test session. You should see that the ls command has been run in window 0!

As the name suggests, the command send-keys sends keypress events to the tmux pane indicated by the -t parameter. This way we can control any application that’s running there from outside.

Now let’s modify those steps slightly to get closer to the goal. In step one, instead run:

tmux new -s test R

This will open the R interpreter in the newly created window. In step two, run:

tmux send-keys -t test:0 'data(iris)' Enter 'hist(Sepal.Width)' Enter

You should see a nice histogram pop up!

Step 2. Sending the Vim selection

We now know how to send keys to an application running inside a tmux pane with a shell command. Now we will learn how to run this command from Vim and pass the currently selected text as parameter.

Vim allows you to run an external command and pass the currently selected lines to its standard input using the :write command. You simply pass the command preceded with a bang sign (!) instead of a filename. Try it by selecting some text in Vim (in visual mode) and running:

:write !tr a b

A new dialog window should pop up, in which you will see the same text with all a’s replaced with b’s. This is the output of the command.

Our goal is to call the tmux send-keys ... command on the selected block of code (say, R code for this example). However, we need to preprocess the code a little bit to ensure that:

each line is followed by Enter,
the whitespaces, quotes and other special characters are passed correctly, instead of breaking the shell command.

Below is the function that does the whole work. You can first add it to your .vimrc and test it a little bit. Then we will discuss it in detail:

function! SendToREPL()
    silent! execute '''<,''>write
      \ !sed "s/.*/\0\nEnter/"
      \ | tr "\n" "\0"
      \ | xargs -0 tmux send-keys -t ' . b:repl_session . ':0.0'
endfunction

In order to test it, you need to set the variable b:repl_session to the name of the tmux session, in which our R interpreter is running:

:let b:repl_session="test"

Then you can call the function by selecting some R code in visual mode and running:

:call SendToREPL()

Note that when you press : in visual mode, the prompt that appears looks like this: :'<,'> - i.e. it contains the selected range. You need to remove those extra characters (up to the colon) to achieve the proper behaviour. We will automatize this in Step 3.

Let’s have a closer look at SendToREPL(). The first line: silent! execute '''<''>write just executes Vim’s write command on the currently selected range. silent! means that we don’t want the dialog window with the output to pop up (because we’re not expecting any output) and the whole command is enclosed in single quotes ('), with quotes inside the command being escaped by doubling them.

The three subsequent lines contain the shell command that is passed to write. It’s actually a pipeline of three commands:

sed "s/.*/\0\nEnter/" adds after each line a line consisting of Enter,
tr "\n" "\0" changes all line breaks to the NUL character - this is important for the next command,
xargs -0 tmux send-keys -t ' . b:repl_session .':0.0' calls the tmux send-keys command while passing the lines from its stdin as arguments. The -0 parameter means that the input lines are separated by NUL characters instead of newlines - this way they are passed literally, no matter what whitespace and special characters they contain. As the target tmux pane, we set the pane 0 of window 0 of the session, the name of which is written in the variable b:repl_session.

Step 3. Ensuring that the interpreter is running

The missing part is now the first Ctrl+E press, which starts the tmux session running the interpreter. In order to simplify this process, we will set some variables based on the file type:

autocmd FileType r,rmd let b:repl_cmd = "R"
autocmd FileType python let b:repl_cmd = "python3"
autocmd FileType r,rmd,python let b:repl_session = "repl-" . b:repl_cmd

The b: prefix means that the variables are local to the buffer. We set two variables: b:repl_cmd is the command to run the appropriate interpreter, while b:repl_session is the name of the tmux session where it’s supposed to be running.

Now we define the following function which starts the interpreter if it’s not already running:

function! EnsureREPL()
    if exists("b:repl_cmd")
        call system("tmux has-session -t " . b:repl_session . 
                    \ " || ( st -e tmux new-session -s " .
                    \ b:repl_session . " " . b:repl_cmd . " ) &")
    endif
endfunction

The tmux has-session command checks whether the session is already running. If not, we start it in a new terminal window - remember to replace st in line 4 with your terminal emulator of choice.

Finally, we set the Ctrl+E keybinding for both normal and visual mode:

autocmd FileType r,rmd,python
  \ nnoremap <C-e> :silent! call EnsureREPL()<CR>
autocmd FileType r,rmd,python
  \ vnoremap <C-e> :<C-u>silent! call SendToREPL()<CR>

Final remarks

As I’ve noted at the beginning, the workflow is not Vim-specific and can be implemented in any text editor that allows selecting a chunk of text and piping it to an external shell command. The essential part here is tmux and this is one example for how powerful this application is when it comes to managing text terminals.

Some might say that there are editors that allow you to run a terminal inside an editor buffer, like neovim. I find this solution somehow inelegant - you typically run a text editor inside a terminal, so you shouldn’t be running another terminal inside the editor. Here we use the editor for editing text and delegate everything else to external applications.

Using this solution for R, I completely eliminated RStudio from my workflow. I don’t miss its constant crashes at all!

Having all PDFs just a few keystrokes away

2020-09-20T00:00:00+03:00

I tend to have a lot of PDF files on my computer: books, science papers, presentations… I often need to quickly open one of them: check a theoretical detail in a book related to what I’m doing or a reference in a paper I’m reading. With the help of dmenu and a short shellscript, I have found a way to make this process as quick and unintrusive as possible.

dmenu: a quick introduction

dmenu is a fantastic program that serves only the purpose of “choosing one option out of a list of possibilities”. It is very similar to fzf that I’ve mentioned before, but operating in a graphical window manager instead of the text terminal. Here’s a minimal example:

echo -e "apples\nbananas\ngrapes\noranges\npears\npineapple" | dmenu

This will show the following menu on the top of the screen:

Once you start typing, the selection will be narrowed down to the entries containing the typed string. The chosen item will be printed on standard output.

The basic solution

In our case, the entries in the list are going to be paths of PDF files on the system. I store all my documents in the directory ~/Documents, so I can obtain a list of all of them with the following command:

find ~/Documents -name '*.pdf'

Let’s combine this with dmenu to be able to choose one file from the list:

find ~/Documents -name '*.pdf' | dmenu -i -l 20

The parameter -i makes dmenu match case-insensitively and -l 20 displays one entry per line in 20 lines (instead of displaying all entries in one line as in the example).

The output of the command is the chosen file. The only missing part now is to pass the filename to your PDF reader of choice. Mine is zathura, so I do the following:

FILENAME=$(find ~/Documents -name '*.pdf' | dmenu -i -l 20) \
&& exec zathura "$FILENAME"

And here’s the result:

Just save it to a shellscript file, bind it to a key in your window manager and there you go!

…except…

If you’re a purist like me, you may be disturbed be the need to call the find command every time this script is executed. If the ~/Documents directory is large, it might be quite slow and involve a lot of unnecessary disk reads. A reasonable solution would be to keep the list of all files in a text file (say, index.txt) and only update it if the content of the directory changes. That’s why we’ll change the script to the following:

if [ ! -f ~/Documents/index.txt ] || [ ~/Documents -nt ~/Documents/index.txt ]; then
  find ~/Documents -name '*.pdf' > ~/Documents/index.txt
fi
FILENAME=$(dmenu -i -l 20 < ~/Documents/index.txt) \
&& exec zathura "$FILENAME"

The condition in the first line makes use of two tests: ! -f is true when the file does not exist, while -nt checks whether the argument on the left is “newer than” the one on the right (in the sense of the “last modified” filesystem entry). Every time you make some modification to the documents directory (like add a new file), its “last modified” date will be updated and this condition will be fulfilled on the next check. In this case, we update the index. This way we can use the index file as input to dmenu and avoid calling find unnecessarily.

It is easy to generalize this script to handle different directories, file extensions and opening programs (movies, images, what have you). This will be left as an exercise for the reader.

Announcing chesstools

2020-05-03T00:00:00+03:00

The long weekend has been very fruitful for me: after some hours of coding, I can proudly announce version 0.1 of chesstools - a suite of minimalist tools for chess study following the UNIX philosophy. The main component of the package is pgnvi - a vi-like editor and browser for PGN files. It comes with a simple graphical application for drawing the chess board and a few other tools. You can get the package from my Gitlab or Github pages.

Recently I was trying to get better in chess, but I’ve always found the educational materials hard to use. Replaying games on the board while reading a book is slow and clumsy, while computer programs for browsing PGN files typically come with terrible bloated GUIs (couldn’t find any counterexample). I noticed that I would need something simple and keyboard-driven, ideally with Vi keybindings, which would integrate well into the terminal-based workflow. It’s just reading text files and drawing a simple picture, shouldn’t be that complicated, right? So I ended up writing it.

Originally I wanted to make a single TUI application, in which the position would be drawn using Unicode symbols for chess pieces (yes, there are such!). It turned out to be an impractical idea, because the resulting board is very small and barely readable. And if you enlarge the terminal font, all the other text in the window becomes too huge.

Then I had this awesome idea: why not delegate the drawing to a separate program that does only this? Simply read the position in FEN notation from stdin (it’s even a single line) and draw it! A text-based PGN browser could then call this program as a subprocess and write positions to its stdin. That’s how drawboard came about on one evening after work.

The main component of the package is pgnvi - a text mode browser and editor for PGN files. It features a tree view that you can use as an opening explorer with large databases. This is how pgnvi’s tree view and drawboard look together in action:

Of course you can also use pgnvi to watch and edit games - all using the vi keybindings:

Instead of an “insert mode”, pgnvi features an “append mode”, in which you can append the moves at the end of the game or delete the moves from a certain position onwards. This is enough for editing - why would you change a chess game in the middle? As the validity of moves and positions is constantly checked, this would quickly result in an invalid state.

The append mode features another interesting functionality. The three main rows of the keyboard (those with letters) are completely remapped to the following layout:

+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+
|  1  |  2  |  3  |  4  |  +  | O-O |  5  |  6  |  7  |  8  |
+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+
 |  a  |  b  |  c  |  d  |     |     |  e  |  f  |  g  |  h  |
 +-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+
  |  R  |  N  |  B  |  Q  |     |  x  |  K  |  B  |  N  |  R  |
  +-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+

Note how the 8 ranks and 8 files used in chess ideally match the 8 fingers that we use for typing (not counting thumbs). The layout corresponds visually to the chessboard and makes the frequently used combinations easily reachable. It will surely have a somewhat steep learning curve, but I’m pretty sure that once mastered, it will make entering moves enormously efficient.

But, I didn’t spend the whole long weekend programming. It’s nice to breathe some fresh air in between. Taking a break between long hours of coding, I did a 20km hike in Finnish woods.

Chesstools is still at an early alpha stage: plagued with bugs, missing important functionality (especially comments and variants!) and exhibiting an ugly and impoverished user interface. But I’ve already used it to transcribe a couple of games - both my own and from books - and to browse a couple of game collections. And it clearly is the chess toolkit that I need. I hope it will be useful for others, too.

And when it achieves maturity, maybe I can finally focus on chess.

Happy new decade!

2020-01-25T00:00:00+02:00

It’s been a while since the last post and for me it was a time of big changes. After having received my PhD, I moved from Leipzig to Helsinki to work as a postdoc at the university here. Of course the relocation, getting used to a new country and starting a new job meant a lot of work and effort, which is why I was too busy to update this site for a while. But everything went more than fine and I’m doing well here up north. As for the site, I have a lot of notes and unfinished articles lying around, so now that I’ve settled in the new place, I might share Linux tips more often. Well, I’m not promising anything, we’ll see.

PhD thesis defended successfully

2019-09-04T00:00:00+03:00

Last month I had my PhD defense at the University of Leipzig and I obtained my PhD title with magna cum laude. In near future, I will be probably moving to another university and widening my research interests, as I am no longer bound to a single, focused long-term goal. If you are interested in my thesis, you can download it here and the code is available on Gitlab.

Change to another shell’s working directory

2019-07-27T00:00:00+03:00

Today I’m going to share another trick for changing a shell’s working directory involving the fuzzy finder. Let us assume that you are doing something in a terminal and then need suddenly need another window for working in the same directory. There is an easy way to do this. It involves defining a function, which shows the current directories of all running shells in fzf, so that we can quickly choose one of them and navigate to it.

I’ll start by showing the full code of the function. Then, I’ll discuss it command by command.

choose_from_cwds() {
	for PROCID in $(pgrep "^$(basename $SHELL)$"); do
		readlink -e /proc/$PROCID/cwd
	done \
	| sort -u \
	| fzf --height=10
}

Of course, I refer to the function presented here by a handy alias:

alias chd='cd "$(choose_from_cwds)" && pwd'

Similarly to the function defined in the previous post, also the alias chd presents me with a fuzzy finder containing a list of directories. When I choose one of them, it navigates to it and displays the directory we’ve end up in.

So now let’s have a look at the commands used in the function.

pgrep "^$(basename $SHELL)$"

This gives us a list of PIDs of all running shell processes. The environment variable $SHELL contains the location of the binary running as shell (e.g. /bin/bash). We extract the name of the binary (bash) using the command basename. Be sure to enclose the name in ^$, so that pgrep does not match other processes whose commands accidentally contain this string.

readlink -e /proc/$PROCID/cwd

This is the most important line of the function: we use the PID of a shell’s process to find out its current working directory. Under Linux, this is very simple: the directory /proc/$PID contains various useful information about the process in question, which are served by the operating system as “files”. One of them is cwd, which is… simply a symbolic link to the process’s current directory! I can’t think of a simpler and more elegant way to expose such information - the beauty of Unix shows here.

We follow the link using the command readlink -e, which returns the path of the referred directory (and also ensures that it exists).

sort -u

Of course we may have multiple shells running with the same current directory. In this case, we don’t want to have multiple identical entries on our list, hence unique sort.

fzf --height=10

Finally, we pipe the list of directories to the fuzzy finder, which presents us with a neat chooser.

Navigate through filesystem quickly with fzf

2019-06-06T00:00:00+03:00

Navigating to the right directory is one of the most unnecessarily time-consuming tasks while working in the shell. Have you ever found yourself typing long paths by hand and repeatedly pressing Tab because you forgot the name of some directory? Personally, I use a couple of tricks to avoid typing paths. Here’s the most important one.

The goal

Let’s say I want to work on the post that you’re currently reading. Normally, I would have to type something like this into my terminal:

$ cd ~/projects/website/
$ cd _posts
$ vim navigate-filesystem-fzf.md

That’s quite a bit of typing! And of course, if I misspell anything, I have to try again. On the other hand, using some tricks, I can get to the right file in just a few keystrokes and I don’t even have to hit exactly the right keys. It looks like this:

Of course, the animation is slowed down a great deal. In my normal workflow, it takes less than 3 seconds. There are a couple of different tricks here, but the most important one is using the program called fzf. Let’s have a closer look.

fzf: a quick introduction

The program you see in action in the above screencast is fzf, which is an abbreviation for “fuzzy finder”. It basically serves one purpose: choosing an item from a list by typing parts of its name. The “item” and the “list” can be whatever, which opens nearly endless possibilities for our creativity.

If you run fzf without any arguments, it serves as a file chooser: it recursively searches all subdirectories of the current directory for files and lets you choose one file. It is worth noting that the recursive search is blazingly fast.

However, the real greatness of fzf comes from its flexibility: if you pass something on standard input, it is interpreted as a list, one item per line. Then, it lets you pick an item from this list, instead of a file. The chosen line is printed on standard output. Try the following:

$ echo -e "apples\nbananas\ngrapes\noranges\npears\npineapple" | fzf

At first, you see the complete list. You can simply navigate it with either Emacs-style (Ctrl-N/P) or Vi-style (Ctrl-J/K) hotkeys. If you start typing, the choice is narrowed to the entries matching what you’ve typed. The matching is - as the program’s name says - “fuzzy”, i.e. not exact. Even if you type non-consecutive letters, like rg, it will still match oranges. It is also tolerant for slight misspellings: apl still selects apples and pineapple.

Navigating through the filesystem

In order to use fzf for navigating, I want to call it in a loop. Every time, it should list the contents of the current directory and let me choose one item from this list. If it is a directory, it should go to this directory and repeat the same. If I’ve chosen a file, it should open the file and terminate. To achieve this behavior, I have written the following function:

fuzzy_open() {
    FILENAME=""
    while FILENAME=$(cat <(find . -maxdepth 1) <(cat "$HOME/.bookmarks") \
                     | fzf --height=10); do
        if [ -d "$FILENAME" ]; then
            cd "$FILENAME";
        else
            break
        fi
    done
    if [ -z "$FILENAME" ]; then
        return 1;
    fi
    ~/.scripts/open "$FILENAME"
}

At its heart lies the following line, which decides which input is piped to fzf:

cat <(find . -maxdepth 1) <(cat "$HOME/.bookmarks")

It is a concatenation of two different sources. find . -maxdepth 1 lists the contents of the current directory. In the second part, I additionally pass entries from a file called .bookmarks in my home directory.

The bookmark file is actually a very important element of the whole thing. It stores paths that I use all the time, like the ones associated with the projects that I’m currently working on. For example, ~/projects/website, which contains my website sources, is listed in the bookmarks, as is ~/Documents/bibliography, where I store all scientific papers.

Every time I find myself in a directory where I’m probably going to return, I simply run:

$ pwd >> ~/.bookmarks

And I have this directory available in a few keystrokes from anywhere. Note that it’s important not to type a single > or bye bye all your bookmarks. It’s probably better to create an alias for this command.

Turning back to fuzzy_open(), once the file name is chosen, there are three possible cases:

It is a name of the directory - change to this directory and repeat.
It is empty - exit with exit code 1. Note that we still end up in the directory we’ve chosen in the previous iteration.
It is a name of a file - open this file. I’ll leave my generic file opening script (~/.scripts/open) for another post, but you can guess what it does.

Finally, in order to be able to call fuzzy_open quickly, I set up a convenient alias:

alias fo='fuzzy_open'

That’s it! I hope you’ve learned something useful and the next time you need to get to the right directory in the shell, you’ll save yourself some time, keystrokes and nerves.

Making your keyboard layout more ergonomic

2019-05-24T00:00:00+03:00

There is no doubt that the keyboard is the most efficient input device to interact with your computer. I use the keyboard for pretty much everything, including web browsing, switching between windows and arranging them, writing e-mails etc. However, many programs require combinations with a modifier key (especially Ctrl) to trigger some functionality. Because of the placement of the modifier keys on the keyboard, repeated pressing of such combinations can be inconvenient and require weird twists and stretches. In this post, I show how to remap certain keys to a much more ergonomic placement.

The home row and the placement of modifier keys

As I live in Germany, I normally use German keyboards with the so-called ISO layout, meaning an extra key between Z and left Shift. However, I use the Polish keyboard layout, which is basically the same as the US layout. Thus, a typical keyboard I work on looks like this:

The keys marked in green are the so-called “home row” - the position in which your fingers should normally rest. They are obviously the easiest keys to reach, as you don’t have to move your fingers at all in order to reach them. Additionally, the keys marked in orange are also quite easy to reach - they require only a slight finger movement.

Now, I mentioned the combinations involving the modifier keys. Let’s have a look on where they are placed:

The both Alt keys are not that bad. They are placed right next to the space bar, so you can press them with your thumbs and the position of the other fingers is almost unaffected. But the other ones, especially the frequently needed Ctrl, are a catastrophe. They are apparently supposed to be pressed by the little finger, which is the weakest of all. Moreover, doing this requires either twisting the wrist outwards, or crooking the little finger, or most often both. This introduces a tension in your hand, requires you to use more strength, and the bend of the wrist changes the position of other fingers, making the letter keys harder to reach. You may laugh, but if you have to do this non-stop for 10 hours a day or longer, it is a serious problem. This is the reason why some people find programs that make extensive use of such combinations, like especially Emacs, unusable.

(I myself don’t use Emacs, but mostly for other reasons. I’ve found that the modkey combinations are not as awful as they seem, if you follow the advice in this post.)

A simple solution

A simple solution to this problem is to place the Ctrl keys near to the home row, so that they are reachable without leaving the natural position:

On the left, pretty much all keyboard have CapsLock in this position. This key is large, wonderfully reachable and completely useless. Remapping it to Ctrl is thus a commonly found advice and I strongly recommend it.

However, I also need an equally convenient right Ctrl key. The key that occupies the right position is the backslash/pipe (\|). It is not used very often, so it doesn’t harm to remap it to Ctrl and find another place for the backslash. On my keyboard, the perfect place for that is the ISO key (marked in blue), which is useless in the Polish/US layout, because its characters - greater/less sign - are also found elsewhere.

That’s it! This slight remapping will make Ctrl-combinations ergonomic. A further idea would be to remap Tab to a modifier key (for example, Super), but it didn’t work well for me. Tab is slightly smaller than CapsLock, so it is not as easy to reach as it seems, and we have a muscle memory for pressing it in some contexts. Moreover, I don’t really need Super, since my window manager (DWM) uses the left Alt as a modifier key.

Some Vim users map Tab to Esc, but for Vim I have a better idea, which I’ll save for another post.

There is also another type of keyboard, called ANSI, on which the Return key is bar-shaped and the backslash is placed above it. I don’t use this type currently and I’m not sure whether remapping backslash to Ctrl would be a good idea there.

xmodmap

On the X window system, such key remapping is ridiculously easy to achieve using the program xmodmap. You describe the desired mappings in a text file, typically called ~/.xmodmaprc, and then simply call:

$ xmodmap ~/.xmodmaprc

It is good to have this command auto-run on system start, for example by putting it into your .xinitrc.

My .xmodmaprc, which does the remapping described above, looks like this:

clear Lock
keycode 66 = Control_L
keycode 51 = Control_R
add Control = Control_L Control_R
keycode 94 = backslash bar backslash bar dead_grave dead_breve

Each physical key on the keyboard is identified by a number (called keycode). Using the keycode command, you can reassign those codes to the “meanings” (keysyms) recognized by X. Remapping CapsLock is a bit tricky, because you have to deactivate the “lock” that it triggers. This is what the first line does. The fourth line adds (or re-adds) the concept of Control modifier, which consists of pressing the left or right Ctrl. I am not sure why it has to be explicitly stated as it should be set up by default, but without this line the “new” Ctrl keys didn’t work.

The last line maps the ISO key to what the backslash key (51) previously was. As you see, you can assign multiple keysyms to a key. The first one is the key pressed without any modifiers, the second one with Shift. The third and fourth one seems to be with some weird modifier key that nobody uses (and with/without Shift), so it is virtually always mapped to the same as the first and second. Finally, the fifth/sixth pair is achieved by additionally pressing AltGr.

Needless to say, the manpage for xmodmap explains all the available commands. It is not a lot in this case.

There are different ways of finding out what the keycode of a key is. Firstly, you can use the xev program. It opens a window and reports all the events for this window (among others, keypresses) to the terminal. If you press a key with this window focused, you will get some information about the pressed key, including its keycode.

Another method is to call:

$ xmodmap -pk

which prints out the table of current mappings.

One final advice

There is a reason why I insisted on having both a left and a right Ctrl in a convenient place. People sometimes tend to use only one of them - typically the left one - and press combinations like Ctrl+S, Ctrl+C with one hand. I think it is a bad habit. I recommend to always press the modifier key with the opposite hand as the letter key. This allows you to use less strength and move your hands less - especially stay in the home row. The same applies to Shift.

Site design finished

2019-05-15T00:00:00+03:00

I’ve been quite busy since the last post, but yesterday and today I finally found time to finish the site. With the nice index of posts by year and by tag on the right (made with jekyll-archive), it is now more or less complete. There is still some content missing on the “Books” and “Software” pages, but I will fill it incrementally. I also already have multiple posts in preparation - I prefer to work on them in parallel, rather than post them one by one. So, real content is going to appear here very soon!