Wiring Forum
http://wiring.org.co/cgi-bin/yabb/YaBB.pl
Programming Questions & Help >> Syntax >> Floating Point Number Precision
http://wiring.org.co/cgi-bin/yabb/YaBB.pl?num=1268758538

Message started by morrowsend on 03/16/10 at 16:55:38

Title: Floating Point Number Precision
Post by morrowsend on 03/16/10 at 16:55:38
I'm working on improving the accuracy of the GPS library (as currently it is about 1,600ft off in my testing.) 

I wrote a converter to convert the lattitude and longitude to the best precision my GPS module gives me, however, it is only output from my wiring board as a float with only 2 digits after the decimal (i.e. 42.00). 

I cannot seem to get more precision to print, even when I try to force it by simply sending a number over serial. I attempted the following:
Serial.print(0.12345) and all I get as a result is "0.12"

I am new to wiring language, so maybe I am completely missing a simple common knowledge thing for this community, and if so, I apologize.

Any help would be appreciated.

Title: Re: Floating Point Number Precision
Post by Titus on 03/16/10 at 20:34:16
What a coincidence, I've been investigating it the day before yesterday.

I didn't got it to print, but you should know that it does contain all the decimals.
So just multiply it by 1000 and make it an int will get you more precision

Titus

Title: Re: Floating Point Number Precision
Post by barragan on 03/17/10 at 00:02:27
which Wiring version and OS are you using?

Title: Re: Floating Point Number Precision
Post by morrowsend on 03/17/10 at 15:43:40
I'm using Wiring version 0026 on Linux Mint 8 (built on Ubuntu Karmic). 

--Thanks Titus for the tip, I'll use that for the moment for quick tests.

Title: Re: Floating Point Number Precision
Post by barragan on 03/18/10 at 02:32:05
there are two versions of print for float or double
Serial.print(floatVar); // the default is for just two decimal digits
Serial.print(floatVar, nDigits); // in this version you can set how many digits to use

Title: Re: Floating Point Number Precision
Post by morrowsend on 03/21/10 at 15:40:54
Thank you very much! That did the trick and now my GPS reading within expected tolerances.

Thanks again!

Title: Re: Floating Point Number Precision
Post by barragan on 03/22/10 at 06:34:38
great :), don't forget to submit your project for the exhibition.

Wiring Forum » Powered by YaBB 2.5 AE!
YaBB Forum Software © 2000-2010. All Rights Reserved.