Results 1 to 4 of 4
 04172013, 12:59 AM #1Member
 Join Date
 Feb 2013
 Posts
 57
 Rep Power
 0
getting lost in double vs. int loss of precision issues...
hi all,
I don 't have good enough planning documentation at my finger tips to get this exactly right. I have an input file and I'm reading it line by line. All lines are either hex or bin numbers. I'm taking them in as a string and I'm doing the math on them by using ints and doubles. They will eventually be printed out in a table using printf(). I'm getting lost in *when* to use and int, when to use a double, does it really matter, and then some. Can someone help me out here? There is only a little bit of this code that is relevant, and I've blocked it out with widthwide comments. thanks!
Java Code:/* The following equations are those that are used in this program for conversions between number bases... * * ANY to DECIMAL: (bit value (or byte value) * (source base ^ bit or byte position)). * DECIMAL to HEX: Decimal value / 16 => remainder appended as new MSB. Do until Decimal value = 0. * DECIMAL to BINARY: Decimal value / 2 => remainder appended as new MSB. Do until Decimal value = 0. * */ import java.io.*; import java.util.*; import java.lang.String; import java.lang.Math; class program_3 { public static void main(String args[]) { //COUNTER VARIABLES int ctr=0; int ctrRev=0; //INPUT VARIABLES String valInput=""; int valLength=0; //CONVERSION VARIABLES String valBin=""; String valHex=""; String valDec=""; double valDecDBL = 0; int baseSource=0; final double base2=2; final double base10=10; final double base16=16; double bitPos=0; int bytePos=0; double valQuotient=0; int valRemainder=0; char bitVal; double bitValDBL = 0; char byteVal; boolean isBinary; FileInputStream fs = new FileInputStream("c:/input.txt"); Scanner sc = new Scanner(fs); //print table headings. System.out.printf("%20s %20s %20s\n", "Binary", "Decimal", "Hexidecimal"); While (sc.hasNextLine()) { valInput = sc.nextLine(); valLength = valInput.length(); //check if binary or hex, use string length = 4 as criteria. if (valLength != 4) { isBinary = true; valBin = valInput; valDec = ""; valHex = ""; } Else { isBinary = false; valHex = valInput; valDec = ""; valBin = ""; } if (isBinary) { //BINARY TO DECIMAL => (bit value (or byte value) * (source base ^ bit or byte position)) valDecDBL = 0; ctrRev = 0; for (ctr = valLength  1; ctr >= 0; ctr) { bitVal = valInput.charAt(ctrRev); bitPos = ctr; if (bitVal == '1') { bitValDBL = 1; valDecDBL = valDecDBL + (bitValDBL * (Math.pow(base2, bitPos))); } ctrRev++; } //DECIMAL TO HEX => (Decimal value / 16) => remainder appended as new MSB. Do until Decimal value = 0. valHex = ""; while (valDecDBL != 0) { valQuotient = Math.floor(valDecDBL / base16); valRemainder = valDecDBL % base16; //loss of precision is occurring here... Switch (valRemainder) { case 15: valHex = "15"; //etc, etc... } } } } } }
thanks!Last edited by ajetrumpet; 04172013 at 01:03 AM.
 04172013, 01:22 AM #2Senior Member
 Join Date
 Jun 2007
 Location
 Bali, Indonesia
 Posts
 762
 Rep Power
 9
Re: getting lost in double vs. int loss of precision issues...
int store an integer numbers while double store floating point numbers. So when you try to assign a double to an int you will loose it precision. double 5.5 can be stored in an int which will only accept the 5 integer and you'll loose the 0.5. Read more on data types here: Primitive Data Types (The Java™ Tutorials > Learning the Java Language > Language Basics)
Website: Learn Java by Examples
 04172013, 03:52 AM #3Member
 Join Date
 Feb 2013
 Posts
 57
 Rep Power
 0
Re: getting lost in double vs. int loss of precision issues...
I realize all of that. What I was really looking for was different ideas on how to rewrite the portion of this code that is causing the problem. thanks.
 04172013, 04:27 AM #4Senior Member
 Join Date
 Jan 2013
 Location
 Northern Virginia, United States
 Posts
 5,024
 Rep Power
 9
Re: getting lost in double vs. int loss of precision issues...
Unless your input data consists of number with decimal points and/or exponents then I would stick with either integer or long values.
Both of these are signed primitives. In terms of binary digits, an integer can have 32. In terms of hex digits, an integer can have 8. Longs have twice as many of both as integers. In both cases the msb (most significant bit) is the sign bit. 1 for negative and 0 for positive.
Regarding your code, I am not certain how you are losing precision. If you keep using the mod function or division with an integer or long then you simply have quotients and remainders. Can you provide an example of how you are losing precision?
Regards,
JimLast edited by jim829; 04172013 at 05:43 AM.
The Java^{TM} Tutorials  SSCCE  Java Naming Conventions
Poor planning on your part does not constitute an emergency on my part
Similar Threads

ERROR : possible loss of precision
By subith86 in forum New To JavaReplies: 1Last Post: 01192011, 06:17 PM 
Possible Loss of Precision?
By jwb4291 in forum New To JavaReplies: 6Last Post: 12022010, 05:28 AM 
Possible loss of precision
By jankidudel in forum New To JavaReplies: 2Last Post: 08072010, 10:15 AM 
possible loss of precision?
By gandalf5166 in forum New To JavaReplies: 6Last Post: 04132010, 08:21 PM 
help me loss of precision error??
By j2vdk in forum New To JavaReplies: 6Last Post: 09012008, 09:23 AM
Bookmarks