In partnership with
Kickstart your holiday campaigns
CTV should be central to any growth marketer’s Q4 strategy. And with Roku Ads Manager, launching high-performing holiday campaigns is simple and effective.
With our intuitive interface, you can set up A/B tests to dial in the most effective messages and offers, then drive direct on-screen purchases via the remote with shoppable Action Ads that integrate with your Shopify store for a seamless checkout experience.
Don’t wait to get started. Streaming on Roku picks up sharply in early October. By launching your campaign now, you can capture early shopping demand and be top of mind as the seasonal spirit kicks in.
Get a $500 ad credit when you spend your first $500 today with code: ROKUADS500. Terms apply.
🚀 Your Investing Journey Just Got Better: Premium Subscriptions Are Here! 🚀
It’s been 4 months since we launched our premium subscription plans at GuruFinance Insights, and the results have been phenomenal! Now, we’re making it even better for you to take your investing game to the next level. Whether you’re just starting out or you’re a seasoned trader, our updated plans are designed to give you the tools, insights, and support you need to succeed.
Here’s what you’ll get as a premium member:
Exclusive Trading Strategies: Unlock proven methods to maximize your returns.
In-Depth Research Analysis: Stay ahead with insights from the latest market trends.
Ad-Free Experience: Focus on what matters most—your investments.
Monthly AMA Sessions: Get your questions answered by top industry experts.
Coding Tutorials: Learn how to automate your trading strategies like a pro.
Masterclasses & One-on-One Consultations: Elevate your skills with personalized guidance.
Our three tailored plans—Starter Investor, Pro Trader, and Elite Investor—are designed to fit your unique needs and goals. Whether you’re looking for foundational tools or advanced strategies, we’ve got you covered.
Don’t wait any longer to transform your investment strategy. The last 4 months have shown just how powerful these tools can be—now it’s your turn to experience the difference.
Detecting unstable markets is not straightforward. However, here we’ll aim to do just that!
We’ll build a “Composite Manipulation Index” to offer a read on how price, volume, and structural changes interact in real time.
We move beyond standard volatility and blend rolling quantiles, median filters, autocorrelation adjustment, and a dedicated volume signal.
Earnings events are automatically masked out to avoid false positives. New market regime shifts get flagged using structural break detection.
The goal isn’t to predict moves, but to identify when the underlying structure departs from the norm.
Especially when unusual volume and persistent patterns line up.
The complete Python notebook for the analysis is provided below.
1. The Composite Manipulation Index
Market manipulation can appear as erratic price swings, sudden volume spikes, or brief regime changes.
The CMI approaches this and flags periods when market structure departs meaningfully from what’s been typical for that asset.
We use price structure, volume, and event-aware filtering. Specifically, the following aspects consitute “market normalcy” as per the our CMI.
Price Structure: Price moves within recent norms, without erratic shifts.
Spectral Balance: Trends and noise are balanced; high-frequency “choppiness” doesn’t dominate.
Predictive Consistency: Recent prices stay close to what short-term patterns suggest.
Volume Activity: Volume stays near its typical range, without outlier spikes or droughts.
Persistence: True instability lasts; signals don’t trigger on brief noise.
Autocorrelation: No abnormal trendiness or mean-reversion versus recent history.
Event Sensitivity: Only assess structure outside scheduled events like earnings.
Regime Stability: No major breaks; broad trends and volatility stay intact.
Retirement Planning Made Easy
Building a retirement plan can be tricky— with so many considerations it’s hard to know where to start. That’s why we’ve put together The 15-Minute Retirement Plan to help investors with $1 million+ create a path forward and navigate important financial decisions in retirement.
1.1 Price Structure
Normal Condition: Price moves without abrupt or outlier shifts.
CMI Methodology: The CMI measures how much price deviates from recent patterns using an adaptive window that expands during high volatility and contracts during calm periods.
BASE_WIN: The minimum window size for all rolling calculations. Larger values smooth out the index; smaller values make it responsive.
ATR_MULT: Multiplier that adjusts how sensitive the adaptive window is to changes in volatility.
1.2 Spectral Balance
Normal Condition: The market maintains a stable blend of slow trends and natural noise, without the market turning “jittery”.
CMI Methodology: The spectral instability metric checks whether high-frequency noise is overtaking smoother market rhythms.
A rise in this ratio suggests more erratic, less organized trading activity.
highBand: The fast EMA deviation to show high-frequency (choppy) price movement.
lowBand: The difference between two slow EMAs to represent longer-term trends.
1.3 Predictive Consistency
Normal Condition: Recent price action does not diverge excessively from its own short-term patterns.
CMI Methodology: The CMI fits a two-term autoregressive model to the closing price, then calculates the error between model and reality.
Elevated error means price is behaving less like itself.
at, bt: Adaptive coefficients, recalculated for each window, that best fit a two-lag model to recent price action.
The adaptive EMA keeps this error measurement up-to-date with changing market regimes.
1.4 Volume Activity
Normal Condition: Trading volume tracks its historical mean, with occasional natural surges for news or trend changes.
CMI Methodology: The CMI uses a rolling z-score to flag when today’s volume is unusually high or low compared to the recent window.
Spikes here, especially alongside price instability, amplify the manipulation signal.
μ_Volume, σ_Volume: The rolling mean and standard deviation of volume.
1.5 Persistence
Normal Condition: Real structural changes last more than one bar; random noise does not.
CMI Methodology: The index only flags high-risk (or clean) zones when instability (or stability) persists for a specified number of consecutive days.
P is the number of consecutive bars required for a signal to be confirmed. Higher persistence reduces false positives from random noise.
1.6 Autocorrelation
Normal Condition: Markets do not show extreme, abnormal trendiness or mean-reversion beyond what’s usual for the regime.
CMI Methodology: Lag-1 autocorrelation of returns is monitored. When autocorrelation is high (either positive or negative), instability readings are adjusted downward to avoid overreacting to benign trends.
Where ρ1,t is the rolling lag-1 autocorrelation of returns.
1.7 Event Sensitivity
Normal Condition: Scheduled events (like earnings releases) are ignored in structural analysis, since they create artificial breaks in price and volume.
CMI Methodology: The CMI masks out readings for dates near earnings or other flagged events.
1.8 Regime Stability
Normal Condition: Broad market structure (e.g. moving average spreads) stays within its usual range.
CMI Methodology: The CMI checks for structural breaks using the rolling standard deviation of the spread between long-term moving averages.
BREAK_Z: The number of standard deviations required to flag a structural break. Larger values mean only extreme shifts trigger a break.
1.9 Composite Scoring
All individual instability measures are blended for a final score:
This raw score is then smoothed with a median filter and an exponential moving average for stability:
And finally, the autocorrelation adjustment is applied:
Signal Generation
Instead of using fixed thresholds, the CMI adapts to recent history via rolling quantiles:
High-risk zones: When CMIadj,t exceeds the recent high quantile and volume is abnormally high, confirmed by persistence.
Clean zones: When CMIadj,t falls below the recent low quantile and volume is quiet, confirmed by persistence.
2. Implementing CMI in Python
2.1. Parameter Setup
We set all the main parameters up front. These control how sensitive or stable the CMI will be.
You can adjust everything, i.e. lookback windows, smoothing, volume thresholds, event padding, and more.
Each parameter is explain as comments in the code snippet below.
import math
import numpy as np
import pandas as pd
import yfinance as yf
import matplotlib.pyplot as plt
from matplotlib.patches import Patch
# ── parameters ──
TICKER = "TSLA"
START_DATE = "2023-01-01"
END_DATE = "2025-12-31"
INTERVAL = "1d"
BASE_WIN = 50 # var window & min EMA window; ↑ smooths variance/EMA more but delays signal, ↓ makes CMI more reactive
ATR_LEN = 14 # ATR lookback; ↑ captures longer volatility trends, ↓ focuses on recent volatility
ATR_MULT = 1.0 # ATR scaling factor; ↑ increases adaptive window size (smoothing), ↓ tightens window (more sensitivity)
SINE_LEN = 20 # sine period; ↑ models longer oscillations, ↓ captures shorter cycles
SMOOTH = 5 # CMI smoothing span; ↑ further smooths final CMI (fewer spikes), ↓ preserves sharp moves
VOL_WIN = 50 # window for volume variance; ↑ uses longer history (stable), ↓ uses short history (sensitive)
VOL_CONFIRM = 1.0 # volume z‐score threshold; ↑ requires stronger volume confirmation, ↓ allows weaker volume support
VOL_SCALE = 0.3 # fraction of price‐axis height to use for max volume bar
MED_FILT = 3 # median filter length; ↑ removes longer noise clusters, ↓ only kills single-bar spikes
PERSIST_DAYS = 2 # days in a row to confirm; ↑ demands longer persistence (fewer signals), ↓ allows quicker signals
QUANT_WINDOW = 20 # days for empirical quantiles; ↑ uses broader regime context, ↓ adapts quicker to new regimes
QUANT_LOW = 0.25 # lower quantile; ↑ relaxes clean threshold (more green zones), ↓ tightens it (fewer green zones)
QUANT_HIGH = 0.80 # upper quantile; ↑ restricts high-risk zone to most extreme (fewer red zones), ↓ broadens risk area
EARN_PAD = 1 # days before/after earnings to mask; ↑ masks larger event window, ↓ only skips exact date
BREAK_WIN = 250 # window for spread std; ↑ smooths break detection (fewer breaks), ↓ reacts faster to regime shifts
BREAK_Z = 3.0 # std‐dev threshold for break; ↑ needs larger deviation to flag break, ↓ flags smaller shifts as breaks2.2. Helper Functions
Next, define a few helper functions to keep the main code simple:
download market price data
fetch earnings dates
calculate the ‘Average True Range’
run adaptive EMAs
estimate rolling autocorrelation
# ── helpers ──
def download_data(tkr):
df = yf.download(
tkr, start=START_DATE, end=END_DATE,
interval=INTERVAL, auto_adjust=True,
progress=False, threads=False
)
if isinstance(df.columns, pd.MultiIndex):
df.columns = df.columns.get_level_values(0)
return df.sort_index().drop_duplicates()
def earnings_days(tkr):
tk = yf.Ticker(tkr)
try:
edf = tk.get_earnings_dates()
if isinstance(edf, pd.DataFrame) and not edf.empty:
return pd.DatetimeIndex(edf.index).normalize()
except Exception:
pass
cal = getattr(tk, "calendar", None)
if isinstance(cal, pd.DataFrame) and "Earnings Date" in cal.index:
return pd.DatetimeIndex([pd.to_datetime(cal.loc["Earnings Date"].iloc[0])]).normalize()
if isinstance(cal, dict):
val = cal.get("Earnings Date") or cal.get("earningsDate")
if val:
return pd.DatetimeIndex([pd.to_datetime(val)]).normalize()
return pd.DatetimeIndex([])
def compute_atr(df, n):
tr = pd.concat([
df["High"] - df["Low"],
(df["High"] - df["Close"].shift()).abs(),
(df["Low"] - df["Close"].shift()).abs(),
], axis=1).max(axis=1)
return tr.rolling(n, min_periods=n).mean()
def adaptive_ema(src, win_s):
out = np.full(len(src), np.nan)
for i in range(len(src)):
w = int(round(win_s.iat[i])) if not math.isnan(win_s.iat[i]) else BASE_WIN
w = max(1, w)
α = 2.0/(w+1.0)
prev = out[i-1] if i>0 else np.nan
out[i] = src.iat[i] if i==0 or math.isnan(prev) else α*src.iat[i] + (1-α)*prev
return pd.Series(out, index=src.index)
def lag1_autocorr(x):
return x.autocorr(lag=1) if x.count()>1 else np.nan2.3. Data Loading and Preparation
Next, pull in the historical price and volume data for the chosen ticker.
Set the main rolling volatility measure, and set up adaptive window lengths for everything else.
# ── load & compute ──
df = download_data(TICKER)
df["ATR"] = compute_atr(df, ATR_LEN)
df["adaptive_win"] = (BASE_WIN*(df["ATR"]/df["Close"])*ATR_MULT).round().clip(lower=10).ffill()2.4. Earnings/Event Calendar
Pull earnings event dates.
These are needed for masking out periods where volatility is about scheduled information releases.
earn = earnings_days(TICKER)2.5. Core CMI Signals
2.5.1 Cycle-Noise Instability
This step tracks how much prices deviate from smooth, cycle-like behavior.
We compare prices to adaptive sine waves to spot when price moves are noisy and structurally unusual.
High values here hint at instability beyond normal random walk noise.
idx = np.arange(len(df))
df["sine"] = np.sin(2*np.pi*idx/SINE_LEN)
df["ema_price"] = adaptive_ema(df["Close"], df["adaptive_win"])
df["ema_sine"] = adaptive_ema(df["sine"], df["adaptive_win"])
df["mse_sine"] = adaptive_ema(
((df["Close"]-df["ema_price"]) - (df["sine"]-df["ema_sine"]))**2,
df["adaptive_win"]
)
df["var_price"] = df["Close"].rolling(BASE_WIN, min_periods=BASE_WIN).var()
df["mi_sine"] = (df["mse_sine"]/df["var_price"]).clip(0,1)2.5.2 Predictive Instability
Now, check how well recent price history predicts the next step.
If simple autoregressive models start failing persistently, it’s a sign that market structure has changed.
df["x1"], df["x2"] = df["Close"].shift(1), df["Close"].shift(2)
for name, ser in {
"sum_x1": df["x1"], "sum_x2": df["x2"], "sum_y": df["Close"],
"sum_x1x1": df["x1"]**2, "sum_x2x2": df["x2"]**2,
"sum_x1y": df["x1"]*df["Close"], "sum_x2y": df["x2"]*df["Close"],
"sum_x1x2": df["x1"]*df["x2"],
}.items():
df[name] = adaptive_ema(ser.fillna(0), df["adaptive_win"])
den = df["sum_x1x1"]*df["sum_x2x2"] - df["sum_x1x2"]**2 + 1e-8
df["a"] = (df["sum_x2x2"]*df["sum_x1y"] - df["sum_x1x2"]*df["sum_x2y"])/den
df["b"] = (df["sum_x1x1"]*df["sum_x2y"] - df["sum_x1x2"]*df["sum_x1y"])/den
df["y_hat"] = df["a"]*df["x1"] + df["b"]*df["x2"]
df["mse_pred"] = adaptive_ema((df["Close"]-df["y_hat"])**2, df["adaptive_win"])
df["mi_pred"] = (df["mse_pred"]/df["var_price"]).clip(0,1)2.5.3 Predictive Instability
We compare slow and fast moving averages.
When short-term price movement (the “high band”) explodes relative to long-term trends, it’s a signal of abrupt regime change.
df["ema34"] = df["Close"].ewm(span=34,adjust=False).mean()
df["ema89"] = df["Close"].ewm(span=89,adjust=False).mean()
df["lowBand"] = df["ema34"]-df["ema89"]
df["highBand"] = df["Close"]-df["Close"].ewm(span=8,adjust=False).mean()
df["energyLow"] = df["lowBand"].rolling(BASE_WIN,min_periods=BASE_WIN).var()
df["energyHigh"] = df["highBand"].rolling(BASE_WIN,min_periods=BASE_WIN).var()
df["mi_spectral"] = (df["energyHigh"]/(df["energyHigh"]+df["energyLow"])).clip(0,1)2.5.4 Volume Instability
Finally, track volume spikes relative to recent history.
Extreme volume changes, especially when paired with the above measures, often mean the market is being pushed or shaken in a non-organic way.
df["vol_z"] = (df["Volume"]-df["Volume"].rolling(60).mean())/df["Volume"].rolling(60).std()
df["var_vol"] = df["vol_z"].rolling(VOL_WIN,min_periods=VOL_WIN).var()
df["mi_volume"] = (df["var_vol"]/(df["var_vol"]+df["var_price"])).clip(0,1)2.6. Composite Scoring and Smoothing
Take the four instability measures, average them, then clean up the signal with a median filter and smoothing.
This produces the core CMI value, which reacts quickly but still avoids overreacting to single-bar noise.
df["cmi_raw"] = (df["mi_sine"] + df["mi_pred"] + df["mi_spectral"] + df["mi_volume"]) / 4
df["cmi_med"] = df["cmi_raw"].rolling(MED_FILT,center=True,min_periods=1).median()
df["CMI_core"] = df["cmi_med"].ewm(span=SMOOTH,adjust=False).mean()2.7. Autocorrelation Adjustment
Markets aren’t always memoryless.
If recent returns are strongly autocorrelated, this code adjusts the CMI so it doesn’t overstate instability just because the market is trending.
The result is a fairer read on true structural shifts.
df["ret"] = df["Close"].pct_change()
df["rho1"] = df["ret"].rolling(20).apply(lag1_autocorr, raw=False)
df["auto_adj"] = np.sqrt((1-df["rho1"])/(1+df["rho1"])).clip(0.1,2)
df["CMI_adj"] = (df["CMI_core"]*df["auto_adj"]).clip(0,1)2.7. Event Masking
Now, mask out any days around scheduled earnings or known events.
mask = df.index.isin(pd.DatetimeIndex(
np.concatenate([pd.date_range(d-pd.Timedelta(days=EARN_PAD), d+pd.Timedelta(days=EARN_PAD)) for d in earn])
))
df.loc[mask, "CMI_adj"] = np.nan2.8. Structural Break Detection
Calculate spread and volatility between slow moving averages.
If the spread jumps outside its recent norm, the model overrides and immediately flags that period as high risk
df["spread"] = df["ema34"] - df["ema89"]
df["spread_std"] = df["spread"].rolling(BREAK_WIN, min_periods=50).std()
df["break_flag"] = (df["spread"].abs() > BREAK_Z * df["spread_std"]).fillna(False)2.9 Adaptive Quantile Thresholds
To adapt to changing regimes, set your high- and low-risk cutoffs using rolling quantiles.
This means the model only calls out the most extreme (or the cleanest) conditions based on what’s normal for the current environment.
df["q_low"] = df["CMI_adj"].rolling(QUANT_WINDOW,min_periods=QUANT_WINDOW).quantile(QUANT_LOW)
df["q_high"] = df["CMI_adj"].rolling(QUANT_WINDOW,min_periods=QUANT_WINDOW).quantile(QUANT_HIGH)2.10. Signal Confirmation and Persistence
Finally, combine the CMI and volume signals and add a persistence filter.
A period is only flagged high-risk or clean if it stays in that state for several days in a row, or if a structural break is present.
high = (df["CMI_adj"] > df["q_high"]) & (df["vol_z"] > VOL_CONFIRM)
low = (df["CMI_adj"] < df["q_low"]) & (df["vol_z"] < -VOL_CONFIRM)
df["sig_hi"] = high.rolling(PERSIST_DAYS).sum() == PERSIST_DAYS
df["sig_lo"] = low.rolling(PERSIST_DAYS).sum() == PERSIST_DAYS
df.loc[df["break_flag"], "sig_hi"] = True2.11. Visualization
Plot everything, i.e. price, volume, the CMI, and the detected risk regimes.
plt.style.use("dark_background")
fig, (ax_p, ax_c) = plt.subplots(
2, 1, figsize=(14, 8), sharex=True, gridspec_kw={"height_ratios": [2, 1]}
)
# price line
ax_p.plot(df.index, df["Close"], color="white", lw=1.2, label="Close")
# compute bar height in price-axis units
ymin, ymax = ax_p.get_ylim()
bar_height = (ymax - ymin) * VOL_SCALE * (df["Volume"] / df["Volume"].max())
bar_bottom = ymin
# volume bars (green if up, red if down)
colors = np.where(df["Close"] >= df["Open"], "green", "red")
width = pd.Timedelta(days=0.8)
ax_p.bar(df.index, bar_height, bottom=bar_bottom,
width=width, color=colors, alpha=0.3, align="center",
label="Volume (scaled)")
# high-risk / clean shading
ax_p.fill_between(df.index, 0, 1, where=df["sig_hi"],
color="red", alpha=0.25,
transform=ax_p.get_xaxis_transform())
ax_p.fill_between(df.index, 0, 1, where=df["sig_lo"],
color="green", alpha=0.25,
transform=ax_p.get_xaxis_transform())
ax_p.set_ylabel("Price")
ax_p.legend(loc="upper left")
# CMI panel
ax_c.plot(df.index, df["CMI_adj"], color="red", lw=1.6, label="CMI")
ax_c.plot(df.index, df["q_low"], "--", color="green", lw=1, label="Low quantile")
ax_c.plot(df.index, df["q_high"], "--", color="orange", lw=1, label="High quantile")
ax_c.scatter(df.index[df["sig_hi"]], df["CMI_adj"][df["sig_hi"]],
marker="o", color="yellow", s=40, label="High-risk")
ax_c.scatter(df.index[df["sig_lo"]], df["CMI_adj"][df["sig_lo"]],
marker="o", color="cyan", s=40, label="Clean")
ax_c.set_ylim(0, 1)
ax_c.set_ylabel("CMI")
ax_c.legend(loc="upper left", ncol=3)
plt.tight_layout()
plt.show()
Figure 1. Composite Manipulation Index for TSLA. Red shading highlights periods of market instability; green marks structurally clean zones confirmed by both price and volume.
3. Interpretation and Fine-Tuning
Red bands and yellow dots mark high-risk periods. These only appear when the CMI stays high and volume is strong for more than a day.
Sometimes, you’ll see high-risk flags if the market’s trend breaks sharply, even if volume is quiet.
Green bands and cyan dots mean the market is unusually stable. The index only calls these when both CMI and volume stay low for several days.
You can make the index more sensitive by lowering the window sizes or persistence.
If you want fewer, bigger signals, raise the thresholds or require longer persistence.
4. Limitations and Improvements
4.1 Limitations
Not All Instability Is Manipulation: CMI flags abnormal structure, not its cause. Many non-manipulative events can trigger high scores.
Parameter Sensitivity: Results depend on parameter choices. There’s no one-size-fits-all setup.
Reliance on Recent History: If market conditions shift, quantile thresholds can lag or misclassify.
Volume Data Limits: In illiquid or synthetic markets, volume can mislead the index.
Imperfect Event Masking: Only known events are masked. Unexpected news still impacts the signal.
4.1 Improvements
Machine-Learned Thresholds: Instead of rolling quantiles, train adaptive models on labeled market regimes to calibrate what’s truly abnormal for each asset.
Order Book Data: Incorporate depth-of-book features (e.g. quote imbalance, order flow toxicity) for more granular detection, especially in intraday settings.
Multi-Asset Context: Add cross-asset or index-relative signals to distinguish between idiosyncratic and market-wide stress.
Real-Time Event Detection: Integrate natural language news feeds or anomaly detection on social signals to flag unscheduled catalysts in real time.
Ensemble Approach: Combine CMI with other statistical indicators (e.g. entropy measures, realized volatility shocks, or market impact metrics) for more robust flagging.
Final Thoughts
The best market tools don’t hand you answers, but instead, they change what you notice.
The CMI gives you a new lens to see risk. If you want to spot regime shifts, this index can help you.