Recursive Vim macros: One step further into automating repetitive tasks

The recent article Use Vim macros to automate frequent tasks by Ricardo Gerardi explored what a macro is and the benefits of using macros to automate repetitive tasks. In this article, you will take a step further and explore what recursive macros are some examples where they might be useful, and some pitfalls that you might run into.

The basic macros workflow consist of record, replay, and profit. When recording, select in which register it will do so. There are registers from a to z available.

[ You might also like: How to use Ansible to configure Vim ]

Once you've opened Vim, to record a macro, start by pressing q and specify the register to save it to. Then type a sequence of commands to be executed and q to end and save the recording. After that, replay the macro by pressing @ and the register.

For example, record a macro like qaA - suffixESCjq, then replay it by pressing @a how many times needed. To make it easier to replay the macro, after running it once, type @@ to replay it.

More about how macro works

Digging into macros a little bit more and explore some of its internals. A macro will stop running in case it can't execute one of its commands. For example, if f( (which means find ( and position the cursor on top of it) can't be executed, then the macro replay will stop immediately. You might think this is a problem, but this is actually a feature, and you can use it in your favor.

When recording a macro, a sequence of commands is entered. That means you can use any command, including the execute macro one (@<register>). This is the essence of recursive macros, calling itself when it runs.

At this point, you could just start recording and replaying some recursive macros. But before you do that, look at one more thing before jumping right into it. When recording a macro, the commands typed get executed as the recording goes. So, if you replay a macro as part of the recording and the related register has something already recorded, it runs and will probably bring unexpected results.

Here comes the first tip: Be sure the register is empty before recording a recursive macro. Clear a register by running q<register>q, which is the same as starting a recording on <register>, running nothing, and ending the recording.

With all that, it's finally time to explore some scenarios where recursive macros will save you extra editing time.

Recording and replaying recursive macros

To start with, let's take the problem solved in the previous article: Convert the following CSV file produced by a hypothetical monitoring system into a /etc/hosts format file.

$ cat hosts.csv
Description,Hostname,IPAddress,FQDN
"OCP Master Node 1",ocpmaster01,192.168.0.20,ocpmaster01.home.ca
"OCP Master Node 2",ocpmaster02,192.168.0.21,ocpmaster02.home.ca
"OCP Master Node 3",ocpmaster03,192.168.0.22,ocpmaster03.home.ca
"OCP Worker Node 1",ocpnode01,192.168.0.30,ocpnode01.home.ca
"OCP Worker Node 2",ocpnode02,192.168.0.31,ocpnode02.home.ca
"OCP Worker Node 3",ocpnode03,192.168.0.32,ocpnode03.home.ca
"Ansible Host 1",ansibleh01,192.168.0.144,ansibleh01.home.ca
"Ansible Host 2",ansibleh02,192.168.0.145,ansibleh02.home.ca

Here's the expected output:

192.168.0.20    ocpmaster01.home.ca    ocpmaster01    #OCP Master Node 1
192.168.0.21    ocpmaster02.home.ca    ocpmaster02    #OCP Master Node 2
192.168.0.22    ocpmaster03.home.ca    ocpmaster03    #OCP Master Node 3
192.168.0.30    ocpnode01.home.ca    ocpnode01    #OCP Worker Node 1
192.168.0.31    ocpnode02.home.ca    ocpnode02    #OCP Worker Node 2
192.168.0.32    ocpnode03.home.ca    ocpnode03    #OCP Worker Node 3
192.168.0.144    ansibleh01.home.ca    ansibleh01    #Ansible Host 1
192.168.0.145    ansibleh02.home.ca    ansibleh02    #Ansible Host 2

The macro that was recorded to help convert the file format is the following:

0r#f"xldf,A,<ESC>p0df,$px:s/,/\t/g<ENTER>j

Assuming that you just opened Vim and the above macro isn't recorded yet, you have some options to populate any register with the macro contents:

1) Record the macro and type all the commands

Start by pressing q then the <register>, type all the commands above and end by pressing q.

2) Set the register

You can set a register using the following command :let @<register>='<commands>' and later use as a macro replay. In this case, you can't run that command directly as the macro has some special keys like Esc and Enter. To work around it, use the :execute command. It allows inputting the special keys as escape sequences. In normal mode, type : and then paste the following line and press Enter:

execute "let @h='0r#f\"xldf,A,\<Esc>p0df,$px:s/,/\\t/g\<Enter>j'"

Make sure the register h contents were set properly. That can be done by running :reg <register> or :reg h in this case:

:reg h
--- Registers ---
"h   0r#f"xldf,A,^[p0df,$px:s/,/\t/g^Mj

The contents of the register are slightly different from what was set by the :execute command. This is because :execute converted \<Esc> and \<Enter> to their key code ^[ and ^M respectively. Those are what Vim uses internally to actually "press" the keys.

At this point, the register h contains the exact same macro recorded on the Use Vim macros to automate frequent tasks article. The next step is to make it recursive. This way, you can trigger it once, and it repeats recursively until it reaches the end of the file.

There is a little trick to edit the already recorded macro and make it recursive. You can also use it to add missing commands and make changes without recording the entire macro.

Start by :let @h=', then Ctrl-r Ctrl-r h, type @h' at the end, and finally press Enter. Ctrl-r Ctrl-r <register> literally inserts the contents of the register. This is the same content displayed when inspecting :reg h.

To check if you have everything, run :reg <register> and check the contents. It should look like the following:

:reg h
--- Registers ---
"h   0r#f"xldf,A,^[p0df,$px:s/,/\t/g^Mj@h

With that in place, go to the second line on hosts.csv and replay the macro @<register> or @h in this example. It will continue until the end of the file updating all the lines. It was possible because the macro had 0 at the beginning and j before recursively replaying itself. That made it go one line down, and when rerunning the macro, it positioned the cursor at the beginning of the line.

Also, observe that the macro went until the end of the file and stopped because it won't be able to go down anymore with j that was our break condition for the recursive execution.

[ Looking for more on system automation? Get started with The Automated Enterprise, a free book from Red Hat. ] 

Wrap up

This was a long journey, but now you have another tool available to help automate repetitive tasks, even if that repetitive task is repeating a macro over and over again.

You also explored editing a macro, so at this point, you might be familiar with recording one from scratch. If you're recording instead of editing a macro, be sure to empty the target register before starting and have @<register> as your last command. Also, think about a breakpoint for your macro. If you're editing until the end of the file, then using j will do it.

Another benefit of a recursive macro is that if something goes wrong, it can be reversed by using a single undo.