• setsubyou@lemmy.world
      link
      fedilink
      English
      arrow-up
      4
      ·
      11 hours ago

      That’s funny because I grew up with math teachers constantly telling us that we shouldn’t trust them.

      Normal calculators that don’t have arbitrary precision have all the same problems you get when you use floating point types in a programming language. E.g. 0.1+0.2==0.3 evaluates to false in many languages. Or how adding very small numbers to very large numbers might result in the larger number as is.

      If you’ve only used CAS calculators or similar you might not have seen these too since those often do arbitrary precision arithmetics, but the vast majority of calculators is not like that. They might have more precision than a 32 bit float though.

    • Pyr@lemmy.ca
      link
      fedilink
      English
      arrow-up
      3
      ·
      edit-2
      14 hours ago

      Only when people use the wrong input, garbage in and garbage out.

      In the same vein I can’t think of any instance where excel had calculated things wrong unless there was a fault in the formula that I made.

      • bus_factor@lemmy.world
        link
        fedilink
        English
        arrow-up
        1
        ·
        4 hours ago

        Except if you’re calculating dates from a long time ago. It famously takes some liberties with leap years.