Advanced

Hard Go with Large Spreadsheets

Story: Review: Hands on LibreOffice 3.3Total Replies: 5
Author Content
robT

Feb 08, 2011
12:11 PM EST
I used LibreOffice spreadsheets to manipulate some large data sets (approx 8000 rows by 29 columns). What I did was use a bunch of vlookups to match dates between two data sets. LibreOffice crashed several times, took 100% of one dual core CPU and consumed 40-50% of 2GB of RAM. The saved spreadsheet was around 10MB. When I broke the problem up between two spreadsheets, CPU use dropped, but memory use went up to 90-100% of RAM (obviously swap was being used and performance dropped accordingly).
herzeleid

Feb 08, 2011
12:19 PM EST
Aside from the observation that this job seems to be perfectly suited to perl, what different spreadsheets have you tried with this?

BTW this discussion brings to mind an office suite called "Applix" that I ran on linux in the 90s - it easily handled large data sets that would crash ms office every time, but unfortunately they are no longer in the market.
jacog

Feb 09, 2011
3:41 AM EST
Hey, I remember Applixware. Quite a decent suite, as I recall. And it's still around:

http://www.vistasource.com/en/products.php

But not too sure how it is these days. :)
herzeleid

Feb 09, 2011
12:55 PM EST
@jacog - thanks for the pointer, who knew? I actually did my first white paper with the current employer back in the 90s, on a solaris-2.5/sparc desktop, running applixware from a remote linux box. Good times...

Edit: Checked out applixware at the link you gave - unfortunately it seems to be stagnant, stuck in a weird time warp when solaris/sparc and windoze peecee were the popular platforms...

Steven_Rosenber

Feb 09, 2011
2:39 PM EST
I'll be working on a 40,000-line spreadsheet soon, and I will report back on how that goes.
bigg

Feb 09, 2011
3:29 PM EST
Use R, not a spreadsheet, if you have 40,000 lines.

Post a reply to this thread:

A free account is needed to post. Write your comment here first: your text is kept in this browser while you sign up or log in, and is waiting for you when you come back.

Your message:
|||||
10
Markdown: **bold**, *italic*, [text](https://link), - lists, > quotes, $x^2$ for math. Formatting help

Sign up and post (it is free)  or  Log in and post