Ex 2: Concatenating multiple feature extraction methods

通用範例/範例二: Concatenating multiple feature extraction methods

在許多實際應用中,會有很多方法可以從一個數據集中提取特徵。也常常會組合多個方法來獲得良好的特徵。這個例子說明如何使用FeatureUnion 來結合由PCAunivariate selection 時的特徵。
這個範例的主要目的: 1. 資料集:iris 鳶尾花資料集 2. 特徵:鳶尾花特徵 3. 預測目標:是那一種鳶尾花 4. 機器學習方法:SVM 支持向量機 5. 探討重點:特徵結合 6. 關鍵函式: sklearn.pipeline.FeatureUnion

(一)資料匯入及描述

    首先先匯入iris 鳶尾花資料集,使用from sklearn.datasets import load_iris將資料存入
    準備X (特徵資料) 以及 y (目標資料)
1
from sklearn.pipeline import Pipeline, FeatureUnion
2
from sklearn.grid_search import GridSearchCV
3
from sklearn.svm import SVC
4
from sklearn.datasets import load_iris
5
from sklearn.decomposition import PCA
6
from sklearn.feature_selection import SelectKBest
7
8
iris = load_iris()
9
10
X, y = iris.data, iris.target
Copied!
測試資料: iris為一個dict型別資料。
顯示
說明
('target_names', (3L,))
共有三種鳶尾花 setosa, versicolor, virginica
('data', (150L, 4L))
有150筆資料,共四種特徵
('target', (150L,))
這150筆資料各是那一種鳶尾花
DESCR
資料之描述
feature_names
4個特徵代表的意義

(二)PCA與SelectKBest

    PCA(n_components = 主要成份數量):Principal Component Analysis(PCA)主成份分析,是一個常用的將資料維度減少的方法。它的原理是找出一個新的座標軸,將資料投影到該軸時,數據的變異量會最大。利用這個方式減少資料維度,又希望能保留住原數據點的特性。
    SelectKBest(score_func , k ): score_func是選擇特徵值所依據的函式,而K值則是設定要選出多少特徵。
1
# This dataset is way to high-dimensional. Better do PCA:
2
pca = PCA(n_components=2)
3
4
# Maybe some original features where good, too?
5
selection = SelectKBest(k=1)
Copied!

(三)FeatureUnionc

    使用sklearn.pipeline.FeatureUnion合併主成分分析(PCA)和綜合篩選(SelectKBest)。
    最後得到選出的特徵
1
# Build estimator from PCA and Univariate selection:
2
3
combined_features = FeatureUnion([("pca", pca), ("univ_select", selection)])
4
5
# Use combined features to transform dataset:
6
X_features = combined_features.fit(X, y).transform(X)
Copied!

(四)找到最佳的結果

    Scikit-learn的支持向量機分類函式庫利用 SVC() 建立運算物件,之後並可以用運算物件內的方法 .fit() 與 .predict() 來做訓練與預測。
    使用GridSearchCV交叉驗證,得到由參數網格計算出的分數網格,並找到分數網格中最佳點。最後顯示這個點所代表的參數
1
svm = SVC(kernel="linear")
2
3
# Do grid search over k, n_components and C:
4
5
pipeline = Pipeline([("features", combined_features), ("svm", svm)])
6
7
param_grid = dict(features__pca__n_components=[1, 2, 3],
8
features__univ_select__k=[1, 2],
9
svm__C=[0.1, 1, 10])
10
11
grid_search = GridSearchCV(pipeline, param_grid=param_grid, verbose=10)
12
grid_search.fit(X, y)
13
print(grid_search.best_estimator_)
Copied!
結果顯示 ``` Fitting 3 folds for each of 18 candidates, totalling 54 fits [CV] featuresuniv_selectk=1, featurespcan_components=1, svmC=0.1 [CV] featuresuniv_selectk=1, featurespcan_components=1, svmC=0.1, score=0.960784 - 0.0s
1
## (五)完整程式碼
2
Python source code: feature_stacker.py
3
http://scikit-learn.org/stable/auto_examples/feature_stacker.html
4
5
```python
6
# Author: Andreas Mueller <[email protected]>
7
#
8
# License: BSD 3 clause
9
10
from sklearn.pipeline import Pipeline, FeatureUnion
11
from sklearn.grid_search import GridSearchCV
12
from sklearn.svm import SVC
13
from sklearn.datasets import load_iris
14
from sklearn.decomposition import PCA
15
from sklearn.feature_selection import SelectKBest
16
17
iris = load_iris()
18
19
X, y = iris.data, iris.target
20
21
# This dataset is way to high-dimensional. Better do PCA:
22
pca = PCA(n_components=2)
23
24
# Maybe some original features where good, too?
25
selection = SelectKBest(k=1)
26
27
# Build estimator from PCA and Univariate selection:
28
29
combined_features = FeatureUnion([("pca", pca), ("univ_select", selection)])
30
31
# Use combined features to transform dataset:
32
X_features = combined_features.fit(X, y).transform(X)
33
34
svm = SVC(kernel="linear")
35
36
# Do grid search over k, n_components and C:
37
38
pipeline = Pipeline([("features", combined_features), ("svm", svm)])
39
40
param_grid = dict(features__pca__n_components=[1, 2, 3],
41
features__univ_select__k=[1, 2],
42
svm__C=[0.1, 1, 10])
43
44
grid_search = GridSearchCV(pipeline, param_grid=param_grid, verbose=10)
45
grid_search.fit(X, y)
46
print(grid_search.best_estimator_)
Copied!