Very Helpful

Story: The trouble with rounding floating point numbersTotal Replies: 4
Author Content
dek

Aug 12, 2006
11:24 AM EDT
Thanks for posting this! I now understand a lot more about floating point error than I did before (and I'm currently a Computer Science student about to Grad). I don't think this has ever come up in my classes (or maybe I wasn't paying attention). If it is the former, it doesn't say much for the quality of the education at the University I'm attending, does it?

Don K.
dinotrac

Aug 13, 2006
5:16 AM EDT
None of this would be news to a fusty old COBOL programmer -- or any mainframe programmer, really.

They would be familiar with packed-decimal (comp-something or other in COBOL) arithmetic, which stores one digit per nibble, just like hex, but does not use a-f, meaning that a packed decimal number used far less space than an EBCDIC (or ASCII) encoded one, but more space than binary encoding.

In mainframe land, the hardware itself supported (still does, I would imagine) packed decimal operations, so they were quite fast.
jdixon

Aug 13, 2006
6:48 AM EDT
> In mainframe land, the hardware itself supported (still does, I would imagine) packed decimal operations,

Ah yes, binary coded decimal. The 6809 microprocessor also had this capability. I have no idea if the Intel chips do or not, as I've never had a need to learn x86 assembly language.
dinotrac

Aug 13, 2006
7:02 AM EDT
jdixon -

I'm amazed at how many CS graduates aren't very familiar with this stuff, especially when you consider that business remains the biggest driver of computing, and those boys and girls do not like to see their finances miscalculated.
jdixon

Aug 13, 2006
6:37 PM EDT
> I'm amazed at how many CS graduates aren't very familiar with this stuff,

Yeah, especially when we covered BCD in my one semester course on microprocessors in my BSEE curriculum. In 1979.

Posting in this forum is limited to members of the group: [ForumMods, SITEADMINS, MEMBERS.]

Becoming a member of LXer is easy and free. Join Us!