Ex 4: Feature Selection using SelectFromModel
此範例是示範以LassoCV來挑選特徵,Lasso是一種用來計算稀疏矩陣的線性模形。在某些情況下是非常有用的,因為在此演算過程中會以較少數的特徵來找最佳解,基於參數有相依性的情況下,使變數的數目有效的縮減。因此,Lasso法以及它的變形式可算是壓縮參數關係基本方法。在某些情況下,此方法可以準確的偵測非零權重的值。
Lasso最佳化的目標函數:
Could not load image
    1.
    LassoCV法來計算目標資訊性特徵數目較少的資料
    2.
    SelectFromModel設定特徵重要性的門檻值來選擇特徵
    3.
    提高SelectFromModel.threshold使目標資訊性特徵數逼近預期的數目

(一)取得波士頓房產資料

1
from sklearn.datasets import load_boston
2
from sklearn.feature_selection import SelectFromModel
3
from sklearn.linear_model import LassoCV
4
5
# Load the boston dataset.
6
boston = load_boston()
7
X, y = boston['data'], boston['target']
Copied!

(二)使用LassoCV功能來篩選具有影響力的特徵

    1.
    由於資料的類型為連續數字,選用LassoCV來做最具有代表性的特徵選取。
    2.
    當設定好門檻值,並做訓練後,可以用transform(X)取得計算過後,被認為是具有影響力的特徵以及對應的樣本,可以由其列的數目知道總影響力特徵有幾個。
    3.
    後面使用了增加門檻值來達到限制最後特徵數目的
    4.
    使用門檻值來決定後來選取的參數,其說明在下一個標題。
    5.
    需要用後設轉換

(三)設定選取參數的門檻值

1
while n_features > 2:
2
sfm.threshold += 0.1
3
X_transform = sfm.transform(X)
4
n_features = X_transform.shape[1]
Copied!

(四)原始碼之出處

Python source code: plot_select_from_model_boston.py
1
# Author: Manoj Kumar <[email protected]>
2
# License: BSD 3 clause
3
4
print(__doc__)
5
6
import matplotlib.pyplot as plt
7
import numpy as np
8
9
from sklearn.datasets import load_boston
10
from sklearn.feature_selection import SelectFromModel
11
from sklearn.linear_model import LassoCV
12
13
# Load the boston dataset.
14
boston = load_boston()
15
X, y = boston['data'], boston['target']
16
17
# We use the base estimator LassoCV since the L1 norm promotes sparsity of features.
18
clf = LassoCV()
19
20
# Set a minimum threshold of 0.25
21
sfm = SelectFromModel(clf, threshold=0.25)
22
sfm.fit(X, y)
23
n_features = sfm.transform(X).shape[1]
24
25
# Reset the threshold till the number of features equals two.
26
# Note that the attribute can be set directly instead of repeatedly
27
# fitting the metatransformer.
28
while n_features > 2:
29
sfm.threshold += 0.1
30
X_transform = sfm.transform(X)
31
n_features = X_transform.shape[1]
32
33
# Plot the selected two features from X.
34
plt.title(
35
"Features selected from Boston using SelectFromModel with "
36
"threshold %0.3f." % sfm.threshold)
37
feature1 = X_transform[:, 0]
38
feature2 = X_transform[:, 1]
39
plt.plot(feature1, feature2, 'r.')
40
plt.xlabel("Feature number 1")
41
plt.ylabel("Feature number 2")
42
plt.ylim([np.min(feature2), np.max(feature2)])
43
plt.show()
Copied!