Feature

Why features matter

A model can only learn patterns that its features actually contain. Relevant features carry a measurable relationship (feature importance) to the outcome being predicted; irrelevant or redundant features add noise and can slow training without improving results.

Features aren't limited to raw data columns. An engineered feature combines or transforms existing data into something more predictive, such as calculating a risk category from a combination of blood pressure and cholesterol level rather than using either value alone.

Feature vs. label

Features and labels sit on opposite sides of a prediction: features are what a model reads, the label is what it's trying to produce. Mixing the two up is a common source of data leakage, covered on the ground truth page.

Types of features

• Raw features — values taken directly from the source data (age, transaction amount, timestamp). Use these when the source data is already predictive on its own.
• Derived features — values calculated from one or more raw features (a risk score, a ratio, a rolling average). Use these when a raw value alone doesn't capture the pattern a model needs.
• Categorical features — values that represent a category rather than a quantity (region, product type, customer segment). Use these to let a model distinguish between groups rather than treat all records identically.

Working with features involves several related processes, each covered on its own page: feature engineering covers building new features from raw data, feature extraction covers reducing data complexity while preserving relevant information, and feature selection covers choosing the subset of features that actually improves a model.

FAQ

What's the difference between a feature and a variable?

In machine learning contexts, the terms are used interchangeably. "Feature" is more common in ML and data science; "variable" is more common in traditional statistics.

What's the difference between a feature and a label?

A feature is an input to a model. A label is the outcome the model is trained to predict. Features come from data you already have; the label is what you're trying to find out.

What makes a feature "good"?

A good feature has a measurable, ideally causal, relationship to the target it's predicting, is available at prediction time (not just during training), and doesn't leak information from the outcome itself. See model evaluation for how feature quality shows up in model performance.