A KMail Breakthrough.

LXer Feature Update: 27-Apr-2016 This tells the story of how I finally managed a successful transfer of email data from KMail version 1.13.6 to version 4.11.5. It is a non-technical essay exploring the obstacles I encountered, my options, and the methods I used to achieve my aim. It was written partly to give the information, but also with the hope that readers will both enjoy and be amused by the story of the "battle of KMail" that was ultimately won against "incredible odds". Links to the earlier articles discussing problems with KMail 4x are given at the end.

Introduction

This article has come about purely because I was downright stubborn - and that is putting it mildly. My current laptop is an old HP Compaq 6710b with a 2-core processor, running a 32 bit Linux system comprising openSUSE 11.4, KDE4.6 and KMail version 1.13.6. Yes, it's antique, but if it does the job that I need, then fine.....if it ain't broke, don't fix it. Unfortunately, while not broken, this dear old laptop is now beginning to really show its age....silly things like keys that may not always work at the moment you need them, or speed, or versions of software that are simply too old to do the job really well any more; just minor irritations, but they all add up.

Last year, I undertook a project with a software development group in Brisbane, and in order to do the work necessary, a new 64 bit laptop was required, so ultimately I found myself the proud possessor of an HP Elitebook 8540P (64bit, 4 core) which was running Windows 7 when I got it. Allow me to confess immediately, that the laptop was NOT brand new......it was a second hand ex-government laptop but in "as-new" condition. And the reason I insisted that it come with Windows 7 was that I knew, for my own ease of use, that the computer would be running the traditional BIOS software chip and therefore I would not have to fight with UEFI. I also made sure that I was able to replace the hard disk drive easily, and that meant that I could remove the Win7 drive and replace it with a blank drive ready to receive Linux. Before someone steps in and says: "Dual Boot", I have never operated that way and don't want to. My preference is to swap the hdd's, and because I know exactly what to do by swapping drives, I prefer to continue that way.

With a blank hdd in place, I loaded the new laptop up with openSUSE 13.1 and proceeded to "play". I chose 13.1 because it is the version that will be supported by the Evergreen project and so 13.1 will retain support for about another two years after SUSE itself stops support. I tried a number of things to get the KMail email files from my old laptop into KMail in this new 64bit machine and in the process of which, altered some settings for KDE. Eventually I found real problems (which I won't discuss here), decided to dump the lot and erased the drive. I then decided that perhaps version 13.2 would be better. Within an hour after installation of 13.2, I regretted that move so much that I once more erased the drive and re-installed 13.1. I also promised myself that under no circumstances would I tamper with any settings involving Akonadi, and I would alter only the minimal ones required to get KMail on line. From then on, I would just simply see what happens when various techniques are tried.

Finally in this introduction, let me say this. I strongly prefer KDE as my workplace and even more strongly prefer KMail as my email client software. I am at home with KDE and I use both the classic menu and the folder view. Once you have those in place, KDE4 closely resembles dear old KDE3.5, but KDE 4 is a far better piece of software apart from its enforced dependence on Akonadi for the "Personal Management Suite and KMail" set of programs - well, that's my opinion. Also, I am now a dedicated Dolphin user as regards file management. I very much like its settings abilities and its split screen mode is wonderful. I'd hate to be without it.

However, in my humble opinion, while KMail is an excellent email client with a superb display, it has been made overcomplex and it can be an absolute "stinker" to work with while you are setting it up if you have never used it before. If you know where to go, you can get the settings for classic style, flat date layout, folder view down the side, etc., but it is exceptionally easy to end up with an extremely messed up display and a "difficult to use" program if you put in the wrong workplace view settings. As for its Akonadi dependence (the Wallet problem appears to have been fixed), words almost fail me as regards this disgusting set of affairs. I retain precisely the same very adverse perceptions that I fully documented in a series on KMail published on LXer a year or so ago. Nevertheless my preference for KMail was such that I soldiered on stubbornly and tried to get this newer version to do what I wanted - and that was not necessarily what the KDE4 and the KMail teams had unilaterally decided all the users wanted. You have to love that arrogance - remove the user's choice fellers, that's the way to go in Linux - NOT !.

Getting the Mail Files Across from KMail 1.13.6 to KMail 4.11.5

The problem seemed insurmountable. KMail 4x can do a simple import of a previously prepared KMail archive file which will contain all your emails and settings etc........BUT.......the much, much older and earlier KMail 1.13.6 does NOT have the ability to archive its files, and therefore you cannot export an archive to be taken up by KMail 4x. So, could you do what used to be so simple with all versions of KMail up to KMail 1.13.6 ?

All you did was make a complete copy of the kmail directory in ..../.kde4/share/apps/kmail/ and drop that kmail copy into the same location in the new installation........KMail immediately recognised all files, folders etc. and away it went. Simple huh ? But you cannot, cannot, cannot do that now.

I can tell you that thanks to the fact that KMail is now integrated with Akonadi, an SQL based server, the email files are apparently saved as part of the Akonadi resources in the location: ....../.local/share/akonadi_maildir_resource_0. But it gets better. If you open that folder, you will find the folders you have just set up on KMail 4x and inside each are the usual "cur", "new" and "tmp" folders. However, my inspection showed that there seems to be a random logic in whether or not the files shown on the KMail display are in any of those folders, or none. Further, that mail directory location is set up as an Akonadi resource, and although it can be handled by the file manager, it does not correspond to the older KMail Unix file system resource. My experience so far strongly suggests that unless an email has passed through the KMail software itself, Akonadi will not know it is there and in turn KMail depends on Akonadi for its recall of email data. I decided that sleeping dogs were best left lying and that discretion was the better part of valour - there had to be an alternative.

(Note: Sadly, what I have written in the paragraph above suggests to me that unlike KMail 1.13.6, where you could so easily use your file manager to make a flawless copy of all your emails perfectly arranged in their respective folders, KMail 4x prohibits this situation. I hope I am wrong, but so far nothing has changed my mind. I suggest some options later.)

Evolution Gives the Answer

It shows how stubbornly frustrated I was getting with the situation in that I also tried two other email client programs, Thunderbird and Evolution. Neither of them appealed to me as much as KMail, although I suppose with time and knowledge and familiarity (especially the last) I am sure I could have grown to enjoy both. In my very early years with Linux, Evolution was my main email software, but that is going way back to around 2003. However, it was Evolution that gave me a key to the problem and it is sad that I cannot acknowledge the writer of this short article, because the name hasn't been included. I had searched on methods of getting KMail files into Evolution, and here is the result that finally "opened the door":

https://eyemeansit.wordpress.com/2011/03/03/migrating-from-k...

As you will see, if you open the article, it is a tutorial which explains the series of short steps that you need to take in order to export the file contents of email folders in KMail into mailbox files (.mbox) that can be imported directly into Evolution. I am not going to repeat the tutorial steps here; they are very clearly written and extremely easy to follow. So, could my old copy of KMail actually do this export ? The answer was very rapidly, "YES", and the process followed the exact steps described in the tutorial written by "eyemeansit". I created a folder called "tempmbox" and eventually I ended up with 23 ".mbox" files of varying sizes which corresponded perfectly to the contents of the 23 folders in my old KMail 1.13.6 software. Naturally, each mbox file was named so that I knew the name of the new folder that would need to be created in KMail 4.11.5, eg. "Crossover.mbox". And of course, there were mbox files corresponding to "inbox.mbox" and "sent-mail.mbox". Okay, that now had things to the point where every email currently held in my old version of KMail had been exported to a series of mbox files that would (or should) be instantly recognisable by both Evolution and hopefully, much later versions of KMail.

Importing into KMail 4.11.5

The next part of the process is tedious, but reasonably simple. You should create the additional folders you need by selecting "Local Folders" in the side panel of KMail 4.11.5, then right click on it and select "Add Folder". It helps if you also have the temporary folder with all the mbox files open in Dolphin (or your preferred file manager) and you can go through that file list one by one while you are adding the names of the KMail folders so that each relevant mbox file will eventually be sent to its same folder name in KMail. Once that is done, you will have your new version of KMail with a set of ready and empty mail folders. I mention here that when importing the .mbox files into KMail, you can send an mbox file into any KMail folder you wish to choose, but in this case, the aim is to duplicate the structure of the original KMail layout.

Now select "File" on the menu bar, and from the drop down menu, select "Import Messages". This will open a subwindow like the one below which was taken from my present installation of KMail 4.11.5. Next, you need to open the selection panel at the top and make sure that the choice of "Import mbox Files (UNIX, Evolution)" is selected from the drop down list. It's right near the top.

{Special Note: it was while editing and checking this article, that for the first time I looked at ALL the options available for importing into KMail 4.11.5, and about halfway down the long list, came across this little gem: "Import KMaildirs and Folder Structure". It may seem odd, but I had never noticed it before because I was searching for an Evolution solution and that option is found virtually at the top of the drop down list - therefore I didn't look further. I have no idea if this option will work correctly on the older KMail structures but at least I do know for certain that the mbox option does. Also, I can only wonder why this particular option specifically for KMail is not at the very head of the list. However, I digress.....back to the battle.}

Next, from the bottom selection panel, open the selection dialogue and select the particular folder in KMail you wish to import email data into. It is at this point that ensuring the names of the mbox files correspond to the KMail folder names makes your task very, very easy. Once you have selected the particular KMail folder into which you want to import data, select "Next". A new subwindow will open which will allow you to select the relevant mbox file.