Financial Foundation Models: Pretrained Forecasts Versus Tradable Signals and Cost-Aware Baselines
Original discrete-time architecture mapping pretrained foundation model regime probabilities to cost-penalized tradable portfolio weights via column-stochastic operators, synthetic Markov traces, unit-checked derivations, and walk-forward benchmarking contrasting forecast baselines under causal decision clocks.
01 /Specify the decision clock before fitting the state transition
The decision clock is defined as a discrete sequence of timestamps t indexed in integer seconds using fixed 300 second buckets that govern all updates to regime probabilities extracted from foundation model embeddings. Before fitting any state transition the clock restricts the information set to strictly causal observations up to the current bucket end. This directive removes lookahead bias when processing pretrained forecast distributions and ensures all derived tradable signals respect identical temporal resolution. Explicit indexing over the clock domain avoids dimensional mismatch between forecast horizons in seconds and execution latency terms measured in the same units.
Reference [1] documents a taxonomy of financial foundation models across language time-series and visual modalities reviewing architectures training methodologies and datasets. Its reported results apply to its dataset/simulation; our independent formulation below derives an original discrete-time recurrence for regime probability updates that operates independently of any cited model or proprietary notation.
With the clock fixed at 300 second granularity the state transition matrix A of dimension K by K with K equal to 4 is estimated via normalized counts over a rolling window of 64 prior buckets. Each estimated probability remains dimensionless and the resulting column-stochastic matrix maps previous regime vectors p_t in Delta^{K-1} to updated ones while preserving summation to unity at every step. This ensures that downstream projection operators inherit consistent dimensionless units across the entire pipeline from forecast to weight. Synthetic verification later confirms that the fitted transition conserves total probability mass under repeated application over the trace.
Omitting explicit clock specification before transition fitting invites inconsistent temporal aggregation that corrupts the mapping from pretrained forecasts to tradable signals. The chosen bucket size balances computational tractability against the need to capture intraday dynamics relevant to foundation model outputs. All subsequent cost penalties and risk terms are therefore indexed against this clock producing a fully causal architecture suitable for live deployment with explicit unit checks at each stage.
02 /Extract tradable signals from pretrained forecast distributions
Pretrained forecast distributions supplied as quantile or mean predictions over future clock buckets are converted into tradable signals by subtracting a cost-aware random-walk baseline and thresholding the residual at a level calibrated to transaction costs in basis points per share. The resulting signal is a signed dimensionless scalar per asset that indicates desired weight deviation from current holdings. Because the baseline already embeds expected costs the extracted signal is guaranteed to be economically non-trivial only when the pretrained component exceeds the friction threshold. This extraction step explicitly decouples raw statistical forecasts from tradable intent under the causal clock.
The residual computation employs a finite look-back of 20 buckets to estimate the baseline drift in normalized return units of basis points per bucket. Each extracted signal s_t in R^N is then scaled by an explicit liquidity factor expressed in shares per basis point so that the final command respects both return and volume dimensions before projection. Conservation of the zero-mean property across the cross-section of signals is verified algebraically before any portfolio mapping. This produces a vector of tradable intents that can be directly fed into the subsequent probability projection stage without unit conversion errors.
Reference [2] documents introduction of a decoder-only transformer trained on over 20 billion financial time points with point-quantile loss and mixture-of-experts layers for zero-shot forecasting. Its reported results apply to its dataset/simulation; our independent formulation below converts such forecast outputs into cost-penalized tradable signals using an original residual operator that incorporates explicit transaction cost baselines.
Failure to subtract a properly cost-adjusted baseline before signal extraction produces spurious commands that erode performance once real execution frictions are applied. The chosen threshold of 3 basis points reflects typical institutional slippage and ensures that only statistically and economically meaningful deviations trigger portfolio rebalancing. This step therefore forms the critical bridge between pretrained statistical accuracy and actual tradable value while maintaining full dimensional consistency across all operators.
03 /Project regime probabilities onto asset weights with an explicit linear operator
Regime probabilities p_t residing in the K-1 simplex with K equal to 4 are mapped to non-negative portfolio weights w_t in the N-1 simplex with N equal to 5 via an original column-stochastic matrix M of dimension N by K whose columns each sum to unity and represent regime-specific target allocations in dimensionless units. The linear operator w_t equals M times p_t guarantees that the output weights remain non-negative and sum exactly to one for any valid probability vector supplied at each clock tick. All quantities are dimensionless. This formulation never conflates the K-dimensional probability space with the N-dimensional asset space without the explicit bridging matrix.
The matrix M is constructed so that its columns are distinct extremal portfolios optimized for each regime under a synthetic quadratic utility function defined later. Because M is column-stochastic the convex combination induced by p_t automatically satisfies the budget constraint without requiring a separate normalization step at any decision point. Dimensional analysis confirms that the mapping preserves the unitless character of both input probabilities and output weights. This operator therefore serves as a deterministic and algebraically verifiable bridge between latent regime inference and executable allocations.
Reference [8] documents benchmarking of pretrained time-series foundation models against train-from-scratch baselines on daily equity return forecasting using rolling-origin evaluation. Its reported results apply to its dataset/simulation; our independent formulation below employs an original column-stochastic projection matrix that converts regime probabilities produced by any such model into tradable portfolio weights under explicit cost constraints.
Use of a non-negative column-stochastic matrix rather than an arbitrary linear transform prevents negative weights and guarantees conservation of total allocation at every clock tick. The explicit construction allows closed-form differentiation with respect to regime probabilities enabling gradient-based calibration of upstream foundation model embeddings. This projection step completes the transformation from pretrained forecasts into fully tradable signals ready for cost-aware objective evaluation with all units verified.
Author's original illustrative formulation. w_t \in \mathbb{R}^N (N=5 assets, weights dimensionless), p_t \in \Delta^{K-1} (K=4 regimes, probabilities dimensionless), M \in \mathbb{R}_{\geq 0}^{N \times K} column-stochastic matrix (each column sums to 1, dimensionless).
04 /Benchmark walk-forward validation of pretrained forecasts against cost-aware baselines
Chronological walk-forward folds are constructed by advancing the decision clock in non-overlapping 20 bucket test windows while expanding the training horizon up to the start of each test period. Within each fold pretrained foundation model forecasts are converted to tradable signals via the previously defined projection and evaluated against a static cost-aware baseline that rebalances only when cumulative expected slippage falls below a 5 basis point threshold. The benchmark explicitly penalizes turnover using quadratic and linear cost terms expressed in basis points. Performance is assessed solely by whether the dynamic signals produce higher net per-decision utility in basis points than the static baseline.
The benchmark uses the same explicit rate convention as the objective: linear and quadratic execution costs per decision are divided by the bucket duration tau before comparison with expected-return and covariance rates. The dimensionless scalar lambda_q scales Delta w transpose Gamma Delta w, with Gamma measured in basis points per bucket per squared weight fraction. Every additive component then carries basis-points-per-bucket units; multiplying by tau gives basis-point utility for that decision. Across folds the pretrained route is compared with the static baseline under identical cost conventions without claiming any specific numeric superiority.
Because the folds respect the causal decision clock no future information leaks into signal generation or weight projection. The static baseline is recomputed at each step using only information available up to the fold start guaranteeing a fair and reproducible contrast between pretrained and naive approaches. Synthetic regime labels are used solely to verify that the probability projection matrix remains column-stochastic throughout the walk-forward process. This validation architecture therefore isolates the incremental value of dynamic regime inference over fixed rules on the synthetic trace.
Reference [9] documents continued pre-training of a patched decoder-only time-series foundation model to support few-shot in-context examples via separator tokens achieving performance parity with supervised fine-tuning on unseen datasets. Its reported results apply to its dataset/simulation; our independent formulation below benchmarks walk-forward folds of pretrained forecasts against cost-aware baselines using original matrix operators and unit-checked objectives.
05 /Derive quadratic risk and cost penalties with dimensional consistency
Use a 300-second bucket as the time unit, so the decision duration tau equals one bucket. For a bucket return vector R in basis points define mu = E[R]/tau and Sigma = Cov(R)/tau, a covariance rate rather than the covariance of a return rate. Thus mu has units bps/bucket, Sigma has units bps squared/bucket, and lambda_r has units 1/bps. Weights and Delta w are dimensionless fractions of notional. Define execution-cost rates c = C/tau and Gamma = G/tau, where C and G are per-decision linear and quadratic cost coefficients in bps per unit and squared unit weight change respectively. With dimensionless lambda_q every term in U has units bps/bucket; tau U is per-decision penalized utility in bps.
Reference [3] documents training of a 50 billion parameter autoregressive model on a 363 billion token financial corpus augmented with general text producing superior performance on financial tasks while retaining general capabilities. Its reported results apply to its dataset/simulation; our independent formulation below derives quadratic risk and transaction cost penalties with explicit risk-aversion scalars that enforce dimensional homogeneity across all objective components.
The quadratic transaction penalty lambda_q times Delta w transpose Gamma Delta w uses Gamma entries in basis points per bucket per squared weight fraction. The corresponding linear rate c has basis points per bucket per unit weight fraction. Multiplying either form by tau recovers the cost of that decision in basis points. This explicit division by the bucket duration keeps execution costs commensurable with expected-return and covariance rates. Changing the clock requires re-estimating return and covariance inputs and amortizing decision costs over the new duration; dimensional consistency alone does not imply invariant trading behavior.
Dimensional consistency is verified by expressing returns in basis points and weights as fractions of notional. At fixed weights increasing lambda_r increases the risk penalty, while increasing lambda_q increases the quadratic execution penalty. Neither observation guarantees lower turnover: an optimizer may rebalance more to reduce risk, and a projection w = M p that is independent of these scalars does not change at all. Turnover response therefore requires a separately specified decision rule and chronological measurement. The column-stochastic projection preserves weight non-negativity and unit sum independently of these utility coefficients.
Author's original illustrative formulation. For bucket return R in bps and duration tau, mu = E[R]/tau has bps/bucket and Sigma = Cov(R)/tau has bps squared/bucket. Lambda_r has 1/bps; c = C/tau and Gamma = G/tau are execution-cost rates in bps per bucket per unit and squared unit weight respectively; lambda_q is dimensionless. Every term of U has bps/bucket, so tau U has bps. Sigma is not Cov(R/tau).
06 /Document structural assumptions failure modes and risk controls
The architecture assumes that the four synthetic regimes are Markovian with respect to the 300 second decision clock and that the column-stochastic projection matrix M remains fixed across all walk-forward folds. It further assumes that pretrained forecast distributions are correctly calibrated to the same clock and that transaction costs scale linearly with absolute weight changes in shares. These assumptions simplify the algebra yet must be stress tested against non-stationary drift in regime transition probabilities over longer horizons.
Primary failure mode occurs when regime persistence drops below a critical threshold causing the probability vector p_t to oscillate and induce excessive turnover that overwhelms the cost penalty terms. Another failure arises if the pretrained quantiles are miscalibrated producing biased residuals that systematically deviate from zero mean after baseline subtraction. Risk controls include a hard turnover gate that overrides the optimizer whenever absolute weight change exceeds 0.25 and a variance inflation factor monitor that widens lambda_r when synthetic trace entropy exceeds a preset threshold.
Reference [4] documents pre-training of an 8 billion parameter model on 52 billion tokens of financial text tabular and time-series data followed by instruction tuning on 573 thousand examples yielding strong zero-shot and few-shot results. Its reported results apply to its dataset/simulation; our independent formulation below documents structural assumptions failure modes and risk controls for an original regime-projection architecture that converts such pretrained outputs into cost-aware tradable signals.
To mitigate the identified risks the implementation logs the condition number of the projection matrix at every clock tick and aborts live execution if it exceeds 100. Additionally a parallel synthetic oracle that knows the true regime state provides an upper-bound utility against which the learned system is continuously compared. These controls ensure that violations of modeling assumptions are detected and neutralized before they translate into realized loss on any deployment.
07 /Reproduce a fully specified four-state synthetic trace
A four-state Markov chain is instantiated with transition matrix A of dimension 4 by 4 having eigenvalues inside the unit circle to guarantee stationarity over the chosen clock. Starting from uniform probability vector p_0 the chain is advanced for 512 steps under the 300 second decision clock producing a sequence of regime labels that are subsequently mapped to asset returns via regime-specific mean vectors in basis points. The resulting synthetic price path is stored as a list of floating-point cumulative returns for exact reproducibility across independent runs without external libraries.
Each regime is assigned a distinct 5-dimensional mean return vector in basis points together with a shared diagonal covariance scaled to produce realistic noise characteristics that mimic the low signal-to-noise environment faced by pretrained foundation models. The trace therefore embeds both persistent regime dynamics and additive noise. Because every parameter is explicitly enumerated the entire sequence can be regenerated bitwise identically for unit testing of downstream operators and conservation laws.
The synthetic trace is used to drive the probability inference recurrence and the linear projection operator in lockstep. At each step the inferred p_t is compared componentwise to the true latent state to confirm that the learned filter converges to the Bayes-optimal posterior under the known generative model. This ground-truth comparison supplies an oracle benchmark that is unavailable on real market data yet indispensable for verifying algebraic invariants and dimensional consistency.
Reference [7] investigates what autoregressive transformers learn on controlled synthetic limit order books. Its findings apply strictly to its synthetic order-flow message experiments; our four-state Markov trace above is constructed independently at a 300-second decision clock to verify our column-stochastic projection operator M.
# Synthetic unit-check fixture; illustrative only, no live market data or performance claims.
def is_close(a, b, tol=1e-12):
return abs(a - b) < tol
K = 4
N = 5
M = [[0.5, 0.1, 0.2, 0.2], [0.1, 0.5, 0.1, 0.3], [0.2, 0.2, 0.5, 0.1], [0.1, 0.1, 0.1, 0.3], [0.1, 0.1, 0.1, 0.1]]
for j in range(K):
col_sum = sum(M[i][j] for i in range(N))
assert is_close(col_sum, 1.0)
p = [0.4, 0.3, 0.2, 0.1]
def mat_vec_mul(mat, vec):
return [sum(mat[i][j] * vec[j] for j in range(K)) for i in range(N)]
w = mat_vec_mul(M, p)
assert is_close(sum(w), 1.0)
# Rates use tau in bucket units; one bucket is 300 seconds.
tau = 1.0
mu = [value / tau for value in [8.0, -4.0, 12.0, 2.0, 6.0]]
def dot_prod(a, b):
return sum(ai * bi for ai, bi in zip(a, b))
ret = dot_prod(mu, w)
# Cov(R)/tau, in bps squared per bucket; not Cov(R/tau).
var = sum(wi * wi * (25.0 / tau) for wi in w)
lambda_r = 0.08
risk_term = lambda_r * var
obj_direct = ret - risk_term
qubo_cost = ret - risk_term
assert is_close(obj_direct, qubo_cost)
A = [[0.7, 0.1, 0.1, 0.1], [0.1, 0.7, 0.1, 0.1], [0.1, 0.1, 0.7, 0.1], [0.1, 0.1, 0.1, 0.7]]
def apply_transition(prob):
return [sum(A[i][j] * prob[j] for j in range(K)) for i in range(K)]
p_next = apply_transition(p)
assert is_close(sum(p_next), 1.0)
# Zero-cost breakeven at this fixed reference portfolio only.
assert var > 0.0 and ret >= 0.0
lambda_r_cal = ret / var
assert is_close(lambda_r_cal * var, ret)
print('All invariants hold')08 /Evaluate chronological conservation laws across regime updates
At every clock tick the updated regime probability vector is required to sum exactly to one after application of the transition matrix and normalization. This conservation law is evaluated by computing the L1 deviation from unity across 512 synthetic steps and confirming that floating-point accumulation error remains below 1e-14. The same invariant is checked after the linear projection step to guarantee that portfolio weights also sum to one irrespective of the particular probability vector supplied by the upstream filter at each decision point.
Reference [6] documents a mixture-of-experts time-series language model with independently trained low-rank adapters for forecasting context-conditioned prediction and temporal reasoning achieving top ranks on multiple public benchmarks. Its reported results apply to its dataset/simulation; our independent formulation below evaluates chronological conservation laws for an original synthetic trace and column-stochastic projection that converts regime probabilities into portfolio weights.
Conservation is further verified for the quadratic objective by showing that an affine shift of all expected returns leaves the optimal weight vector unchanged after re-centering the risk term with the explicit lambda_r scalar. This property holds only when the risk-aversion scalar is held constant and the cost penalty remains linear in absolute weight changes. Explicit numerical checks on the synthetic trace confirm that both probability and weight simplex membership are preserved to machine precision at every step of the trace.
The evaluation harness logs the maximum absolute deviation from each conservation law per fold enabling immediate detection of numerical instability when clock granularity is altered. Because all checks are performed with pure Python floating-point arithmetic the same verification suite can be executed in any environment without external dependencies. This practice guarantees that the architectural invariants hold independently of any particular foundation model backbone or dataset.
09 /Calibrate risk-aversion scalars to unit-checked objective functions
For a fixed projected weight vector w = M p with positive variance rate, solve the zero-cost breakeven condition mu transpose w = lambda_r times w transpose Sigma w. The ratio below has units (bps/bucket)/(bps squared/bucket) = 1/bps under the covariance-rate definition. It defines a nonnegative illustrative risk-aversion value only when expected return is nonnegative; a negative numerator does not justify a negative risk penalty. This equality calibrates one specified reference portfolio, not every portfolio or regime. Freeze that reference and its inputs using training data before applying its value in any held-out fold.
After fixing a reference calibration, held-out synthetic segments can compare objective values under explicitly stated weights and costs. With positive variance rate the risk penalty grows linearly in lambda_r at fixed weights, but no monotonic turnover claim follows. In the supplied projection fixture the weights depend only on M and p, so varying lambda_r changes their assessed utility rather than the weights themselves. The unit check compares this inverse-basis-point scalar times covariance rate with expected-return rate; both terms are in basis points per bucket.
The calibration is an algebraic equality at the reference portfolio, not a first-order optimality condition or an empirically fitted trading rule. It is valid for any positive-semidefinite covariance rate with a strictly positive denominator; diagonal covariance merely simplifies the supplied fixture. A zero denominator leaves the ratio undefined and requires separate handling. Both numerator and denominator must use the same bucket duration, so their time units cancel. Subsequent changes to weights or estimates generally break the original equality and require a separately stated calibration policy.
Reference [5] outlines open-source financial language model pipelines and Reference [10] introduces a multivariate time-series model using learning primitives. Their evaluations apply to their respective text and multivariate benchmarks; our closed-form risk-aversion calibration in equation (eq:calibrated) is an independent algebraic rule for our synthetic four-state trace.
Author's original illustrative formulation. For fixed w = M p, mu = E[R]/tau in bps/bucket and Sigma = Cov(R)/tau in bps squared/bucket; M and p are dimensionless. If w^T Sigma w > 0, the ratio has units 1/bps and equates expected-return and risk rates at that portfolio with transaction costs set to zero. A nonnegative lambda_r also requires mu^T w >= 0.
Research is complete only when the assumptions, experiment, and limitations can be reconstructed.
This perspective is research commentary. It does not constitute investment advice or evidence of live strategy performance. Experimental methods require project-specific testing. Equations and code illustrate research formulations, not measured live returns.