Floating-point engine

Deep dive: Calculation engine — ×, ÷, ^, roots, the transcendentals (sin/cos/ln/eˣ), and number formatting.

All TI-BASIC arithmetic runs through a BCD floating-point engine centered on the OP registers in RAM. The engine lives mostly on flash page 0 (it’s hot), with the RST-30 shortcut for the most common op.

Number format — TIFloat (9 bytes on disk) [confirmed]

+0  type      0x00 = real (positive), 0x80 = negative real;
              0x0C/0x8C = complex (paired with the imaginary part)
+1  exp       base-10 exponent, biased by 0x80 (0x80 = 10^0)
+2..+8  mantissa   7 bytes = 14 packed BCD digits, normalized d.ddddddddddddd

As a C struct:

typedef struct {
    uint8_t type;          /* +0: 0x00 real (positive), 0x80 negative; 0x0C/0x8C complex part */
    uint8_t exp;           /* +1: base-10 exponent, biased by 0x80 (0x80 == 10^0)             */
    uint8_t mantissa[7];   /* +2..+8: 14 packed BCD digits, normalized d.ddddddddddddd         */
} TIFloat;                                              /* 9 bytes on disk / in a stored var   */
/* In an OP register slot the number occupies 11 bytes: the 9 above plus 2 trailing guard      */
/* digit bytes (OP1EXT at +9/+10) used during math — see "OP registers" below.                 */

The stored value is

$$v = \pm\,(d_0.d_1d_2\cdots d_{13})\times 10^{\,e-\mathtt{0x80}}$$

where $e$ is the biased exponent byte and $d_0\ldots d_{13}$ are the 14 BCD mantissa digits. For a normalized nonzero value, $d_0$ is nonzero; zero is a special representation. This is decimal precision, not a binary IEEE format.

The guarded raw-byte scan finds 144 overlapping nine-byte windows that satisfy the type, exponent, packed-digit, and normalization heuristic. Greedily skipping nine bytes after each match leaves 134 candidates. These counts include instruction bytes and unrelated data. Counts derived from Ghidra data definitions instead depend on listing state and do not measure the same ROM-only candidate set. [confirmed]

The constants used below are confirmed separately by Ghidra references and raw ROM bytes. Examples include $\pi/180 = 1.745\ldots\mathrm{e}{-2}$, $180/\pi = 5.729\ldots\mathrm{e}{1}$, 65536, and the Flash page 02 transcendental coefficient tables. [confirmed]

OP registers — 11 bytes each [confirmed]

OP1–OP6 begin at 0x8478 and occupy 11 bytes each. Their shared layout is:

typedef struct {
    TIFloat value;
    uint8_t guard[2];
} TIOpRegister;

The guard bytes extend the stored BCD mantissa during calculation. OP1 is the primary accumulator. The real add/subtract/multiply/divide entries use OP2 as their second operand and return in OP1; complex operations and comparisons have distinct register and flag contracts.

Core operations [confirmed]

The four basic real arithmetic entries have the shape OP1 ∘ OP2 → OP1. Add and subtract align exponents; multiply and divide instead combine them. The sign is handled separately from the BCD digit work. For nonzero inputs, negation toggles type bit 7; _InvOP1S also clears the sign of zero. The add/subtract helpers process eight bytes: seven stored mantissa bytes plus the first extension byte, giving 16 working decimal digits. The second extension byte is cleared too, but is not part of that eight-byte add loop. The following flow describes add/subtract, not every binary operation:

flowchart LR
    A["clear guard digits<br/>fp_clear_guard"] --> B["align exponents<br/>shift smaller right by Δ"]
    B --> C["BCD digit op<br/>add / subtract"]
    C --> D["renormalize<br/>back to d.dddd…"]
    D --> E["round on guards<br/>write type/exp to OP1"]

The page-0 entry points — the hottest get a one-byte RST shortcut, which is why FP code is dense with RST 30h/08h/20h:

Alignment, then the worked example — _FPAdd

To combine $x=(-1)^{s_x} m_x\times 10^{e_x}$ and $y=(-1)^{s_y} m_y\times 10^{e_y}$, the engine first aligns to the larger exponent. With $e_x \ge e_y$ it shifts $m_y$ right by

$$\Delta = e_x - e_y \quad(\text{digit shifts})$$

one nibble per fp_shift_right_digit call; if $\Delta > 15$ the smaller operand falls entirely past the 14 mantissa digits plus the 2 guard digits and is dropped. It then adds the aligned mantissas when the signs match ($s_x = s_y$) and subtracts when they differ ($s_x \ne s_y$), fixing the result’s sign afterward — the essence of sign-magnitude arithmetic:

\begin{algorithm}
\caption{\texttt{\_FPAdd}: $OP1 \gets OP1 + OP2$ (sign-magnitude BCD)}
\begin{algorithmic}
\IF{$OP2 = 0$}
    \RETURN $OP1$
\ENDIF
\IF{$OP1 = 0$}
    \STATE $OP1 \gets OP2$ \COMMENT{incl. extended bytes}
    \RETURN $OP1$
\ENDIF
\STATE $\Delta \gets \mathrm{exp}(OP1) - \mathrm{exp}(OP2)$ \COMMENT{\texttt{fp\_exp\_diff}}
\IF{$|\Delta| > 15$}
    \RETURN larger operand \COMMENT{other is negligible}
\ENDIF
\STATE shift the smaller mantissa right by $|\Delta|$ digits to align \COMMENT{\texttt{fp\_shift\_right\_digit}}
\IF{$\mathrm{sign}(OP1) = \mathrm{sign}(OP2)$}
    \STATE $\mathrm{mantissa} \gets$ BCD-add
\ELSE
    \STATE $\mathrm{mantissa} \gets$ BCD-subtract
    \STATE fix result sign \COMMENT{\texttt{fp\_sub\_mantissa}}
\ENDIF
\STATE round via the guard digits, renormalize, store exp/type in $OP1$
\RETURN $OP1$
\end{algorithmic}
\end{algorithm}

The full helper cluster is documented below.

Dynamic confirmation. Traced under headless TilEm: the 2+3 run (home-2plus3.macro) enters _FPAdd and — signs equal — falls through the sign test to fp_add_mantissa (ram:1cb9), while the 5−2 run (fpsub.macro) negates OP2 and takes the opposite-sign branch into fp_sub_mantissa (ram:1d37). fp_sub_mantissa has 0 hits in the add trace and the add path 0 hits in the subtract trace, so the pseudocode’s sign dispatch is confirmed both ways.

The FP helper cluster [confirmed]

These five page-0 primitives are shared by add/sub/mult/div and the transcendentals. All were decompiled and disassembled in this ROM; the fp_* names below are the project’s labels (in tools/symbols/names.txt). They operate on the OP-register guard region (OP1EXT/OP2EXT are 2 bytes each, at 0x8481–0x8482/0x848C–0x848D — fp_clear_guard zeroes all four) and the 7-byte mantissas of OP1 / OP2 (OP1M 0x847A / OP2M 0x8485, two bytes past the type/exponent bytes at 0x8478/0x8483).

ram:1d2f and ram:1d37 are two entry points into the same BCD-subtract body — 1d2f loads HL=0x8481 (OP1 guard), DE=0x848C (OP2 guard) and computes OP2 − OP1 into OP2 (LD A,(DE)
SUB (HL)), while 1d37 enters with the pointers swapped for the reverse OP1 − OP2, before joining the common loop — so the caller picks the subtraction direction by choosing the entry. This is what lets _FPAdd produce a non-negative magnitude and then fix the sign.

Multiply, divide, and the transcendental routines on Flash page 02 reuse the same alignment and normalization primitives.

Accumulator high-nibble helper [confirmed]

_ShRAcc = 0x41D4, body ram:1BCB, is a six-instruction scalar helper rather than an OP-register operation. It executes four RRA instructions, masks with 0x0F, and returns the original high nibble of A in the low nibble. The final AND defines the returned flags; no other register is touched.

A controlled trace passes A = 0xAB and records A = 0x0A, F = 0x1C after the bcall returns. The result is in tools/data/community-bcall-semantics.csv. [confirmed] under TilEm.

Floating-point stack (FPS) [standard]

FPS (0x9824) is a software stack for temporaries; _PushRealO1 (= RST 18h, ram:155C), _PushReal, _PopRealO1 through _PopRealO6, _PopReal, _AllocFPS, and _DeallocFPS manage it. Used to spill OP registers during nested expression evaluation.

Multiplication, division, and transcendentals [confirmed]

The rest of the FP operation set lives alongside addition on page 00; the transcendental routines are banked to Flash page 02:

See Calculation engine for the ×/÷/^/root algorithms and number formatting.

Transcendental method [confirmed]

The ln/e^x/sin-cos evaluators use local page 02 code and coefficient tables. fp_mul_indexed_constant (ram:2362) calls the stub at ram:3DD1, whose inline descriptor 1E 7D 02 selects coeff_fetch (02:7D1E). It then enters the _FPMult body at ram:2392. The preceding LD A,n selects a coefficient; it does not select a flash page. The actual banked-call helper is cross_page_jump (ram:2B09). [confirmed]

The shared algorithm — digit-by-digit pseudo-division [confirmed]

The table paths in the forward log and exp evaluators use a digit-by-digit pseudo-division recurrence. logexp_digit_table (02:7181) contains the 16 values $\log_{10}(1+10^{-k})$ for $k=0\ldots15$. Each step scales a BCD value by $1+10^{-k}$ with one digit shift and one BCD addition. Only base conversion uses fp_mul_indexed_constant and general multiplication within these table paths. Logarithms also have a near-one path using division, described below. The traces also separate the accumulator entries: _EToX uses fp_add_mantissa (ram:1CB9), while _LnX uses its sibling at ram:1CA9. [confirmed]

Logarithm. With the exponent already split off so the mantissa is $x\in[1,10)$, the loop (02:6F80–6FEE) drives $x$ up toward $10$ by repeatedly scaling by the largest table factor that doesn’t overshoot. The weighted sum of the table entries approximates $\log_{10}(10/x)$; the repetition counts are not decimal digits of the logarithm.

\begin{algorithm}
\caption{Logarithm by pseudo-division (table $c_k=\log_{10}(1+10^{-k})$ at \texttt{02:7181})}
\begin{algorithmic}
\REQUIRE reduced mantissa $x_0 \in [1,10)$, working value $x \gets x_0$, accumulator $L \gets 0$
\FOR{$k = 0$ \TO $15$}
    \WHILE{$x \cdot (1+10^{-k}) \le 10$}
        \STATE $x \gets x + (x \gg k\text{ digits})$ \COMMENT{$\times(1+10^{-k})$ is a BCD shift-add}
        \STATE $L \gets L + c_k$
    \ENDWHILE
\ENDFOR
\RETURN $\log_{10}x_0 \approx 1 - L$ \COMMENT{$x$ driven toward $10$, base conversion multiplies by $\ln 10$}
\end{algorithmic}
\end{algorithm}

The two passes split the coarse digits ($k=0\ldots7$) from the fine digits ($k=8\ldots15$). fp_constant_table (02:7D42) supplies $\ln 10$ as row 6 through fp_mul_indexed_constant; _LogX skips that final multiply.

Exponential. _EToX/_TenX (02:7066+) reverse the recurrence, consuming the fractional part $y$ by subtracting $\log_{10}(1+10^{-k})$ while building a product of the factors $1+10^{-k}$. Both paths visit table selectors in increasing order.

\begin{algorithm}
\caption{Exponential by pseudo-multiplication (inverse recurrence)}
\begin{algorithmic}
\REQUIRE $y_0 = $ fractional part of $x\log_{10}e$, residual $y \gets y_0$, accumulator $A \gets 1$
\FOR{$k = 0$ \TO $15$}
    \WHILE{$y \ge c_k$}
        \STATE $y \gets y - c_k$
        \STATE $A \gets A + (A \gg k\text{ digits})$ \COMMENT{$\times(1+10^{-k})$}
    \ENDWHILE
\ENDFOR
\RETURN $10^{y_0} \approx A$
\end{algorithmic}
\end{algorithm}

logexp_digit_table powers ln, log, eˣ, and 10ˣ. fp_constant_table supplies the base-conversion and trig-reduction constants. [confirmed]

Dynamic confirmation. Traced under headless TilEm: ln(2) (ln2.macro) drives _LnX, whose selector (02:6F94) steps A=00…07 then 08…0F (the coarse/fine split at the 6FAD AND 0x8 / 6FD3 BIT 4 tests), walking successive 02:7181 rows with a per-step shift-add, then fetches $\ln 10$ via LD A,6
CALL ram:2362 and multiplies. e^{1} (exp1.macro) drives _EToX, which consumes the inverse recurrence with increasing table indices (the inner step is fp_sub_mantissa 1d37, the accumulator add fp_add_mantissa 1cb9), selector sweeping 00…0F under the 710A CP 0x0F bound. On-screen results: .6931471806 and 2.718281828.

_SinCosRad uses the same recurrence shape on the range-reduced angle. trig_recurrence_table_a (02:7201) and trig_recurrence_table_b (02:7281) each contain eight rows with two sign/phase variants selected by OP5.value.type bit 7. The exact rotation identity encoded by each row remains open. [confirmed]

_LnX — natural log (02:6EFD) [confirmed]

The real-input _LnX entry first calls _CkOP1Pos (ram:1E5D) and rejects a negative sign. The shared core separately rejects zero at 02:6F1E through ram:212D; the sign helper alone does not reject zero or check the type. The core at 02:6F1B selects between two paths. Near one, 02:6F33–6F7C forms (x-1)/(x+1), rescales it, calls the shared digit recurrence at 02:77A6 with A=0x80, and doubles its result. This branch bypasses logexp_digit_table; it corresponds to the identity $\ln x=2\operatorname{atanh}((x-1)/(x+1))$. The table path splits $x$ into mantissa and exponent. Its pseudo-division loop at 02:6F8C–6FEC steps through logexp_digit_table. The first phase stops when the selector reaches bit 3; the second stops at bit 4. Calls to fp_mul_indexed_constant select row 3 for $\log_{10}e$ and row 6 for $\ln 10$. [confirmed]

_EToX — eˣ (02:705C) [confirmed]

_EToX clears the guard digits, then uses fp_mul_indexed_constant row 3 for $\log_{10}e$. Its JR at 02:7064 skips _TenX’s guard initialization and joins the shared body at 02:7069. That body splits the integer digit shift, handles sign and reciprocal cases, then evaluates the fractional part through logexp_digit_table. The CP 0x0F bound at 02:7109 establishes 16 selector slots. [confirmed]

_SinCosRad sine and cosine in radians (02:733E) [confirmed]

This routine keeps its range reduction on Flash page 02:

1. Mode/select flags. 0x8499 holds the trig-op selector — 0x01 (sin), 0x02 (cos), 0x04 (tan) — ORed with 0x80 when (IY+0) bit 2 is clear (BIT 2,(IY+0)
JR NZ,+2
OR 0x80). _SinCosRad itself enters with A=0x81, so it stores 0x81 regardless. fp_clear_guard and _ZeroOP3 initialize the work area.

2. Exponent gate. LD A,(0x8479)
SUB 0x80
JP C,02:73D4
CP 0x0C
JP NC,ram:26F4 — magnitudes below 1 skip the initial full-period reduction. Decimal exponents ≥ 12 raise DOMAIN directly. The bound is ROM-confirmed; a formal reduction-error bound is not established here.

3. Reduce the angle. It reduces against the stored period constants and takes the fractional part to find the quadrant. The reduction constants are the page-0x02 BCD block:

   • 02:7D81 — the 2π full-turn modulus (mantissa 62 83 18 53 07 17 96 = 6.2831853…), copied to the OP3 work reg via LD HL,02:7D81
CALL ram:1AE2 (ram:1AE2/copy7_from_8490 copies 7 mantissa bytes to 0x8490).
   • 02:7D8E, 02:7D95, 02:7D96 — companion constants used in the quadrant-fixup / remainder comparisons (CALL ram:1D7B magnitude compare at 02:73B1/02:7447). The quadrant (0–3) is accumulated in B/bStack_1 (bits 0/3/6) and decides sin-vs-cos and the result sign (the XOR 0x1 / OR 0x8 / XOR 0x8 flag juggling at 02:7424–02:7464).

4. Per-digit evaluation. The reduced argument enters trig_hyperbolic_recurrence (02:7489), distinct from the log/exp table loops. For sin(1), the reduced argument is $r = \pi/2 - 1 = 0.5707963267948966$. The engine computes $\cos r$, while the quadrant bits in OP5.value.type carry the sign and phase. The recurrence has three phases; the first two consume the trig tables:

   • Phase 1 — digit extraction (02:74A4–02:74E0). For rows $k=0,\ldots,7$, 02:74A8 sets DE = OP2M and calls the table-A entry at 02:731D. The selected row address is 0x7201 + 16*k + 8*v, where $v$ is bit 7 of OP5.value.type. ram:1A94 copies the eight bytes. fp_align_round_diff aligns the row into OP3 at scale $10^{-(k+1)}$. A non-restoring subtract/add sweep reduces the accumulator modulo 1 and stores one decimal digit in OP5.value.mantissa[k]. For sin(1), the digits are 6,3,8,8,2,4,3,6, leaving $u \approx 2.56\times10^{-9}$. [confirmed]
   • Phase 2 — correction product (02:74E6–02:7528). Entry 02:7312 loads trig_recurrence_table_b[0], where $b_0 = 0.9509852944837202$, into OP2M. At each digit position $k$, the loop performs $n_k = \lfloor(11-d_k)/2\rfloor$ BCD shift-add steps of $\mathtt{OP2} \gets \mathtt{OP2} + \mathtt{OP2}\cdot10^{-2(k+1)}$. The INC B at 02:74F8 precedes the doubled shift count at 02:7504–7506. This builds $b_0\cdot\prod_k(1+10^{-2(k+1)})^{n_k}$ without general multiplication. For sin(1), the product is $0.9704891777365256$. [confirmed]
   • Phase 3 — result assembly (02:752A onward). The engine walks the stored digits again with align/add-sub steps and exponent bookkeeping. For sin(1), it produces OP4 = 0.8414709848078931, with cos(1) = 0.5403023058681400 alongside. [confirmed]

   The closed-form identity of the phase-1 digit map remains open; see Open questions.

Dynamic confirmation. Traced under headless TilEm: sin(1) in radian mode (sin1.macro) drives _SinCosRad. The flag init, the exponent gate (735D LD A,(0x8479)
SUB 0x80
JP C,02:73D4
CP 0x0C
JP NC — neither branch taken, since the decimal exponent of 1 is 0), and the period-constant load from 02:7D81 (7372 LD HL,02:7D81
CALL ram:1AE2). The trace records all three recurrence phases and the on-screen result .8414709848. It also records eight phase-1 entries at 02:731D, with B = 0–7 and HL = 0x7201 + 16·B + 8 after each ram:1A94 copy, followed by one 02:7312 entry for the phase-2 base row.

Coefficient tables [confirmed]

coeff_fetch zeroes OP2.value.type and indexes fp_constant_table[A] with a nine-byte stride. Rows 0–6 contain one exponent byte and eight packed-BCD mantissa bytes: 16 decimal digits, with no stored type byte. The copy at 02:7D39 plus two LDIs actually reads ten bytes into OP2+1 through OP2+10, so the second extension byte receives the byte immediately following the nine-byte coefficient. For row 6, that byte is the first direct trig mantissa byte at 02:7D81, not another exponent-prefixed row. The first nine copied bytes supply the coefficient and first guard byte. The only LD A,n
CALL fp_mul_indexed_constant uses in this cluster select row 3 (log10(e)) and row 6 (ln(10)). Later trig reduction constants are loaded directly from the same block.

02:7D42 constants, exponent + eight BCD bytes, 9-byte stride:
  [00] 81 57 29 57 79 51 30 82 32
  [01] 80 15 70 79 63 26 79 48 97
  [02] 7F 78 53 98 16 33 97 44 83
  [03] 7F 43 42 94 48 19 03 25 18  ; log10(e) fetch site
  [04] 80 31 41 59 26 53 58 98 00
  [05] 7E 17 45 32 92 51 99 43 30
  [06] 80 23 02 58 50 92 99 40 46  ; ln(10) fetch site

The following bytes at 02:7D81 belong to directly loaded trig-reduction mantissas, not further exponent-prefixed rows in this constant table.

Three entry paths share the indexing tail at 02:7320. Entry 02:7301 selects logexp_digit_table and OP4M. Entry 02:7312 selects trig_recurrence_table_b and OP2M. Entry 02:731A selects OP4M, then falls through 02:731D to select trig_recurrence_table_a. The phase-1 path enters at 02:731D with OP2M already in DE. The shared tail adds 16*B + 8*v to HL, where $v$ is bit 7 of OP5.value.type, then ram:1A94 copies the eight-byte row to the destination. [confirmed]

logexp_digit_table has 16 eight-byte rows:

[00] 30 10 29 99 56 63 98 12  [01] 04 13 92 68 51 58 22 50
[02] 00 43 21 37 37 82 64 26  [03] 00 04 34 07 74 79 31 86
[04] 00 00 43 42 72 76 86 27  [05] 00 00 04 34 29 23 10 45
[06] 00 00 00 43 42 94 26 48  [07] 00 00 00 04 34 29 44 60
[08] 00 00 00 00 43 42 94 48  [09] 00 00 00 00 04 34 29 45
[10] 00 00 00 00 00 43 42 94  [11] 00 00 00 00 00 04 34 29
[12] 00 00 00 00 00 00 43 43  [13] 00 00 00 00 00 00 04 34
[14] 00 00 00 00 00 00 00 43  [15] 00 00 00 00 00 00 00 04

The two trig recurrence tables hold the forward sin/cos near-unity factors. Each row is 16 bytes: the first eight-byte variant is selected when OP5.value.type bit 7 is clear, and the second when it is set.

02:7201:
[00] 09 96 68 65 24 91 16 20 | 10 03 35 34 77 31 07 56
[01] 09 99 96 66 68 66 65 24 | 10 00 03 33 35 33 34 76
[02] 09 99 99 96 66 66 68 67 | 10 00 00 03 33 33 35 33
[03] 09 99 99 99 96 66 66 67 | 10 00 00 00 03 33 33 33
[04] 09 99 99 99 99 96 66 67 | 10 00 00 00 00 03 33 33
[05] 09 99 99 99 99 99 96 67 | 10 00 00 00 00 00 03 33
[06] 09 99 99 99 99 99 99 97 | 10 00 00 00 00 00 00 03
[07] 10 00 00 00 00 00 00 00 | 10 00 00 00 00 00 00 00

02:7281:
[00] 95 09 85 29 44 83 72 02 | 10 52 06 69 51 89 55 92
[01] 99 94 95 10 19 99 69 80 | 10 00 50 52 03 08 13 30
[02] 99 99 94 94 95 10 20 35 | 10 00 00 50 50 52 03 05
[03] 99 99 99 94 94 94 95 10 | 10 00 00 00 50 50 50 52
[04] 99 99 99 99 94 94 94 95 | 10 00 00 00 00 50 50 51
[05] 99 99 99 99 99 94 94 95 | 10 00 00 00 00 00 50 51
[06] 99 99 99 99 99 99 94 95 | 10 00 00 00 00 00 00 51
[07] 99 99 99 99 99 99 99 95 | 10 00 00 00 00 00 00 01

The table-driven ln, e^x, and sin/cos paths advance one coefficient-table row at a time with shift-and-add. Ln/e^x use logexp_digit_table; sin/cos use the two trig recurrence tables. The inverse-trig arctangent engine instead uses a base-10 CORDIC iteration, documented in Calculation engine.