Series Filtering

Remove series that don't meet quality criteria for forecasting.

Function	Description
drop_constant	Remove series with no variation
drop_short	Remove series below minimum length
drop_gappy	Remove series exceeding max gap ratio
drop_zeros	Remove all-zero series

Showing 4 of 4

---

anofox_fcst_ts_drop_constant

Removes series exhibiting no variation (constant values). Constant series cannot be forecasted meaningfully.

Parameters

Parameter	Type	Required	Description
table_name	VARCHAR	Yes	Source table
group_col	ANY	Yes	Group column
value_col	DOUBLE	Yes	Value column

Output

Returns filtered table excluding constant series.

Example

-- Remove products with no sales variation
SELECT * FROM anofox_fcst_ts_drop_constant(
    'sales_by_product',
    product_id,
    sales
);

warning

May drop intermittent demand series with many zeros if gaps haven't been filled first. Fill gaps before filtering to preserve sparse series.

---

anofox_fcst_ts_drop_short

Filters out series shorter than a minimum length threshold. Short series don't have enough history for reliable forecasting.

Parameters

Parameter	Type	Required	Description
table_name	VARCHAR	Yes	Source table
group_col	ANY	Yes	Group column
min_length	INTEGER	Yes	Minimum number of observations

Output

Returns filtered table excluding short series.

Example

-- Keep only products with at least 30 observations
SELECT * FROM anofox_fcst_ts_drop_short(
    'sales_by_product',
    product_id,
    30
);

Minimum Length Guidelines

Model Type	Recommended Minimum
Naive / SMA	10 observations
Exponential Smoothing	20 observations
ARIMA	30 observations
Seasonal models	2+ full seasons
TBATS / MSTL	3+ full seasons

warning

May drop intermittent demand series if gaps haven't been pre-filled. Fill gaps before filtering.

---

anofox_fcst_ts_drop_gappy

Removes series where the proportion of missing values exceeds a threshold. Useful for filtering out series with too many gaps for reliable forecasting.

Parameters

Parameter	Type	Required	Description
table_name	VARCHAR	Yes	Source table
group_col	ANY	Yes	Group column
value_col	DOUBLE	Yes	Value column
max_gap_ratio	DOUBLE	Yes	Maximum gap ratio (0-1). Series with more gaps are removed

Output

Returns filtered table excluding gappy series.

Example

-- Remove series with more than 20% missing values
SELECT * FROM anofox_fcst_ts_drop_gappy(
    'sales_by_product',
    product_id,
    sales,
    0.2
);

---

anofox_fcst_ts_drop_zeros

Removes series that contain only zero values. All-zero series provide no signal for forecasting.

Parameters

Parameter	Type	Required	Description
table_name	VARCHAR	Yes	Source table
group_col	ANY	Yes	Group column
value_col	DOUBLE	Yes	Value column

Output

Returns filtered table excluding all-zero series.

Example

-- Remove products with zero sales across all periods
SELECT * FROM anofox_fcst_ts_drop_zeros(
    'sales_by_product',
    product_id,
    sales
);

---

Filtering Workflow

-- 1. Fill gaps first (important!)
CREATE TABLE sales_filled AS
SELECT * FROM anofox_fcst_ts_fill_gaps(
    'sales_by_product',
    product_id,
    date,
    sales,
    '1d'
);

-- 2. Remove constant series
CREATE TABLE sales_variable AS
SELECT * FROM anofox_fcst_ts_drop_constant(
    'sales_filled',
    product_id,
    sales
);

-- 3. Remove short series
CREATE TABLE sales_ready AS
SELECT * FROM anofox_fcst_ts_drop_short(
    'sales_variable',
    product_id,
    30
);

-- 4. Check how many series remain
SELECT COUNT(DISTINCT product_id) as remaining_products
FROM sales_ready;

---

Pre-Filtering Analysis

Before filtering, understand what you'll be removing:

-- Check series lengths
SELECT
    product_id,
    COUNT(*) as length,
    CASE
        WHEN COUNT(*) < 30 THEN 'Too short'
        ELSE 'OK'
    END as status
FROM sales_by_product
GROUP BY product_id
ORDER BY length;

-- Check for constant series
SELECT
    product_id,
    STDDEV(sales) as std_dev,
    CASE
        WHEN STDDEV(sales) = 0 THEN 'Constant'
        ELSE 'Variable'
    END as status
FROM sales_by_product
GROUP BY product_id;

---