Leaf-Wise Tree Growth
20 min
XGBoost's default strategy grows trees LEVEL-WISE: expand every leaf at
the current depth before going any deeper, keeping the tree roughly
balanced. LightGBM's signature difference is LEAF-WISE growth: instead
of expanding a whole level at once, always split whichever SINGLE leaf
(anywhere in the tree, at any depth) would reduce error the most next.
This tends to reach a given error level with fewer total splits — but
can produce deeper, more lopsided trees, which is why LightGBM's
num_leaves (a direct cap on total leaves) is its primary complexity
control, rather than max_depth.
from lightgbm import LGBMRegressor
def train_and_predict(X_train, y_train, X_test, num_leaves):
model = LGBMRegressor(n_estimators=20, num_leaves=num_leaves, random_state=0, verbosity=-1, min_child_samples=1)
model.fit(X_train, y_train)
return [round(float(p), 4) for p in model.predict(X_test)]
num_leaves is doing the same job max_depth did for XGBoost -- capping how complex each tree is allowed to get -- just measured directly in leaf count rather than depth, which fits leaf-wise growth's asymmetric tree shapes more naturally than a depth limit would.
Write `train_and_predict(X_train, y_train, X_test, num_leaves)`: train an `LGBMRegressor(n_estimators=20, num_leaves=num_leaves, random_state=0, verbosity=-1, min_child_samples=1)`, fit it, and return predictions on `X_test` as a list, each rounded to 4 decimal places.
XGBoost's default tree growth is LEVEL-WISE (expand every leaf at the current depth before going deeper). LightGBM grows LEAF-WISE instead. What's the difference?
You can train a LightGBM model and control its complexity via num_leaves, and understand how leaf-wise growth differs structurally from XGBoost's level-wise default.