用於非線性可分資料的支援向量機

目錄

• 目標
• 動機
• 最佳化問題的擴充套件
• 原始碼
• 解釋
• 結果

上一教程： 支援向量機簡介
下一教程： 主成分分析（PCA）簡介

原始作者	Fernando Iglesias García
相容性	OpenCV >= 3.0

目標

在本教程中，您將學習如何

• 定義當訓練資料無法線性分離時SVM的最佳化問題。
• 如何配置引數以使SVM適應這類問題。

動機

為什麼擴充套件SVM最佳化問題以處理非線性可分離的訓練資料很有趣？計算機視覺中大多數使用SVM的應用都需要比簡單線性分類器更強大的工具。這是因為在這些任務中，訓練資料很少能使用超平面進行分離。

以其中一項任務為例，例如人臉檢測。在這種情況下，訓練資料由一組人臉影像和另一組非人臉影像（世界上除人臉以外的所有其他事物）組成。這些訓練資料過於複雜，無法找到每個樣本的表示（特徵向量），從而使整個人臉集與整個非人臉集線性可分。

最佳化問題的擴充套件

請記住，使用SVM我們獲得一個分離超平面。因此，由於訓練資料現在是非線性可分離的，我們必須承認找到的超平面將錯誤分類一些樣本。這種錯誤分類是最佳化中的一個新變數，必須予以考慮。新模型必須同時包含尋找能提供最大邊距的超平面這一舊要求，以及透過不允許過多分類錯誤來正確泛化訓練資料這一新要求。

我們從這裡開始闡述尋找最大化邊距的超平面的最佳化問題（這在之前的教程中有所解釋（支援向量機簡介））

\[\min_{\beta, \beta_{0}} L(\beta) = \frac{1}{2}||\beta||^{2} \text{ 滿足條件 } y_{i}(\beta^{T} x_{i} + \beta_{0}) \geq 1 \text{ 對於所有 } i\]

有多種方法可以修改此模型，使其考慮誤分類錯誤。例如，可以考慮最小化相同的量加上一個常數乘以訓練資料中的誤分類錯誤數量，即

\[\min ||\beta||^{2} + C \text{(誤分類誤差)}\]

然而，這不是一個很好的解決方案，因為，除其他原因外，我們沒有區分與正確決策區域距離較小或不小的錯誤分類樣本。因此，一個更好的解決方案將考慮錯誤分類樣本與其正確決策區域的距離，即

\[\min ||\beta||^{2} + C \text{(錯分樣本到其正確區域的距離)}\]

對於訓練資料的每個樣本，都定義了一個新引數 \(\xi_{i}\)。這些引數中的每一個都包含其對應訓練樣本到其正確決策區域的距離。下圖顯示了來自兩個類的非線性可分訓練資料、一個分離超平面以及被錯誤分類樣本到其正確區域的距離。

注意

圖中僅顯示了被錯誤分類樣本的距離。其餘樣本的距離為零，因為它們已經位於其正確的決策區域中。

圖中出現的紅色和藍色線條是到每個決策區域的邊距。非常重要的是要認識到，每個 \(\xi_{i}\) 都從一個錯誤分類的訓練樣本延伸到其相應區域的邊距。

最後，最佳化問題的新公式是

\[\min_{\beta, \beta_{0}} L(\beta) = ||\beta||^{2} + C \sum_{i} {\xi_{i}} \text{ 滿足條件 } y_{i}(\beta^{T} x_{i} + \beta_{0}) \geq 1 - \xi_{i} \text{ 且 } \xi_{i} \geq 0 \text{ 對於所有 } i\]

引數 C 應該如何選擇？顯然，這個問題的答案取決於訓練資料的分佈方式。雖然沒有通用的答案，但以下規則很有用：

• 較大的 C 值會給出更少的誤分類錯誤但更小的邊距的解決方案。請注意，在這種情況下，誤分類錯誤代價高昂。由於最佳化的目標是最小化引數，因此只允許少量誤分類錯誤。
• 較小的 C 值會給出更大的邊距和更多的分類錯誤的解決方案。在這種情況下，最小化不會過多地考慮求和項，因此它更側重於尋找具有大邊距的超平面。

原始碼

你也可以在OpenCV原始碼庫的samples/cpp/tutorial_code/ml/non_linear_svms資料夾中找到原始碼，或者從這裡下載。

C++

• 可下載程式碼：點選此處
• 程式碼速覽

  #include <iostream>
  #include <opencv2/core.hpp>
  #include <opencv2/imgproc.hpp>
  #include "opencv2/imgcodecs.hpp"
  #include <opencv2/highgui.hpp>
  #include <opencv2/ml.hpp>
   
  using namespace cv;
  using namespace cv::ml;
  using namespace std;
   
  static void help()
  {
  cout<< "\n--------------------------------------------------------------------------" << endl
  << "This program shows Support Vector Machines for Non-Linearly Separable Data. " << endl
  << "--------------------------------------------------------------------------" << endl
  << endl;
  }
   
  int main()
  {
  help();
   
      const int NTRAINING_SAMPLES = 100; // Number of training samples per class
      const float FRAC_LINEAR_SEP = 0.9f; // Fraction of samples which compose the linear separable part
   
      // Data for visual representation
      const int WIDTH = 512, HEIGHT = 512;
      Mat I = Mat::zeros(HEIGHT, WIDTH, CV_8UC3);
   
      //--------------------- 1. Set up training data randomly ---------------------------------------
      Mat trainData(2*NTRAINING_SAMPLES, 2, CV_32F);
      Mat labels (2*NTRAINING_SAMPLES, 1, CV_32S);
   
      RNG rng(100); // Random value generation class
   
      // Set up the linearly separable part of the training data
      int nLinearSamples = (int) (FRAC_LINEAR_SEP * NTRAINING_SAMPLES);
   
      // Generate random points for the class 1
      Mat trainClass = trainData.rowRange(0, nLinearSamples);
      // The x coordinate of the points is in [0, 0.4)
      Mat c = trainClass.colRange(0, 1);
  rng.fill(c, RNG::UNIFORM, Scalar(0), Scalar(0.4 * WIDTH));
      // The y coordinate of the points is in [0, 1)
  c = trainClass.colRange(1,2);
  rng.fill(c, RNG::UNIFORM, Scalar(0), Scalar(HEIGHT));
   
      // Generate random points for the class 2
  trainClass = trainData.rowRange(2*NTRAINING_SAMPLES-nLinearSamples, 2*NTRAINING_SAMPLES);
      // The x coordinate of the points is in [0.6, 1]
  c = trainClass.colRange(0 , 1);
  rng.fill(c, RNG::UNIFORM, Scalar(0.6*WIDTH), Scalar(WIDTH));
      // The y coordinate of the points is in [0, 1)
  c = trainClass.colRange(1,2);
  rng.fill(c, RNG::UNIFORM, Scalar(0), Scalar(HEIGHT));
   
      //------------------ Set up the non-linearly separable part of the training data ---------------
      // Generate random points for the classes 1 and 2
  trainClass = trainData.rowRange(nLinearSamples, 2*NTRAINING_SAMPLES-nLinearSamples);
      // The x coordinate of the points is in [0.4, 0.6)
  c = trainClass.colRange(0,1);
  rng.fill(c, RNG::UNIFORM, Scalar(0.4*WIDTH), Scalar(0.6*WIDTH));
      // The y coordinate of the points is in [0, 1)
  c = trainClass.colRange(1,2);
  rng.fill(c, RNG::UNIFORM, Scalar(0), Scalar(HEIGHT));
   
      //------------------------- Set up the labels for the classes ---------------------------------
  labels.rowRange( 0, NTRAINING_SAMPLES).setTo(1); // Class 1
  labels.rowRange(NTRAINING_SAMPLES, 2*NTRAINING_SAMPLES).setTo(2); // Class 2
   
      //------------------------ 2. Set up the support vector machines parameters --------------------
  cout << "Starting training process" << endl;
      Ptr<SVM> svm = SVM::create();
  svm->setType(SVM::C_SVC);
  svm->setC(0.1);
  svm->setKernel(SVM::LINEAR);
  svm->setTermCriteria(TermCriteria(TermCriteria::MAX_ITER, (int)1e7, 1e-6));
   
      //------------------------ 3. Train the svm ----------------------------------------------------
  svm->train(trainData, ROW_SAMPLE, labels);
  cout << "Finished training process" << endl;
   
      //------------------------ 4. Show the decision regions ----------------------------------------
      Vec3b green(0,100,0), blue(100,0,0);
      for (int i = 0; i < I.rows; i++)
      {
          for (int j = 0; j < I.cols; j++)
          {
              Mat sampleMat = (Mat_<float>(1,2) << j, i);
              float response = svm->predict(sampleMat);
   
              if (response == 1) I.at<Vec3b>(i,j) = green;
              else if (response == 2) I.at<Vec3b>(i,j) = blue;
          }
      }
   
      //----------------------- 5. Show the training data --------------------------------------------
      int thick = -1;
      float px, py;
      // Class 1
      for (int i = 0; i < NTRAINING_SAMPLES; i++)
      {
  px = trainData.at<float>(i,0);
  py = trainData.at<float>(i,1);
  circle(I, Point( (int) px, (int) py ), 3, Scalar(0, 255, 0), thick);
      }
      // Class 2
      for (int i = NTRAINING_SAMPLES; i <2*NTRAINING_SAMPLES; i++)
      {
  px = trainData.at<float>(i,0);
  py = trainData.at<float>(i,1);
          circle(I, Point( (int) px, (int) py ), 3, Scalar(255, 0, 0), thick);
      }
   
      //------------------------- 6. Show support vectors --------------------------------------------
  thick = 2;
      Mat sv = svm->getUncompressedSupportVectors();
   
      for (int i = 0; i < sv.rows; i++)
      {
          const float* v = sv.ptr<float>(i);
          circle(I, Point( (int) v[0], (int) v[1]), 6, Scalar(128, 128, 128), thick);
      }
   
      imwrite("result.png", I); // save the Image
      imshow("SVM for Non-Linear Training Data", I); // show it to the user
      waitKey();
      return 0;
  }

Java

• 可下載程式碼：點選此處
• 程式碼速覽

  import java.util.Random;
   
  import org.opencv.core.Core;
  import org.opencv.core.CvType;
  import org.opencv.core.Mat;
  import org.opencv.core.Point;
  import org.opencv.core.Scalar;
  import org.opencv.core.TermCriteria;
  import org.opencv.highgui.HighGui;
  import org.opencv.imgcodecs.Imgcodecs;
  import org.opencv.imgproc.Imgproc;
  import org.opencv.ml.Ml;
  import org.opencv.ml.SVM;
   
  public class NonLinearSVMsDemo {
      public static final int NTRAINING_SAMPLES = 100;
      public static final float FRAC_LINEAR_SEP = 0.9f;
   
      public static void main(String[] args) {
          // 載入本地 OpenCV 庫
  System.loadLibrary(Core.NATIVE_LIBRARY_NAME);
   
  System.out.println("\n--------------------------------------------------------------------------");
  System.out.println("This program shows Support Vector Machines for Non-Linearly Separable Data. ");
  System.out.println("--------------------------------------------------------------------------\n");
   
          // Data for visual representation
          int width = 512, height = 512;
  Mat I = Mat.zeros(height, width, CvType.CV_8UC3);
   
          // --------------------- 1. Set up training data randomly---------------------------------------
  Mat trainData = new Mat(2 * NTRAINING_SAMPLES, 2, CvType.CV_32F);
  Mat labels = new Mat(2 * NTRAINING_SAMPLES, 1, CvType.CV_32S);
   
  Random rng = new Random(100); // Random value generation class
   
          // Set up the linearly separable part of the training data
          int nLinearSamples = (int) (FRAC_LINEAR_SEP * NTRAINING_SAMPLES);
   
          // Generate random points for the class 1
  Mat trainClass = trainData.rowRange(0, nLinearSamples);
          // The x coordinate of the points is in [0, 0.4)
  Mat c = trainClass.colRange(0, 1);
          float[] cData = new float[(int) (c.total() * c.channels())];
          double[] cDataDbl = rng.doubles(cData.length, 0, 0.4f * width).toArray();
          for (int i = 0; i < cData.length; i++) {
  cData[i] = (float) cDataDbl[i];
          }
  c.put(0, 0, cData);
          // The y coordinate of the points is in [0, 1)
  c = trainClass.colRange(1, 2);
  cData = new float[(int) (c.total() * c.channels())];
  cDataDbl = rng.doubles(cData.length, 0, height).toArray();
          for (int i = 0; i < cData.length; i++) {
  cData[i] = (float) cDataDbl[i];
          }
  c.put(0, 0, cData);
   
          // Generate random points for the class 2
  trainClass = trainData.rowRange(2 * NTRAINING_SAMPLES - nLinearSamples, 2 * NTRAINING_SAMPLES);
          // The x coordinate of the points is in [0.6, 1]
  c = trainClass.colRange(0, 1);
  cData = new float[(int) (c.total() * c.channels())];
  cDataDbl = rng.doubles(cData.length, 0.6 * width, width).toArray();
          for (int i = 0; i < cData.length; i++) {
  cData[i] = (float) cDataDbl[i];
          }
  c.put(0, 0, cData);
          // The y coordinate of the points is in [0, 1)
  c = trainClass.colRange(1, 2);
  cData = new float[(int) (c.total() * c.channels())];
  cDataDbl = rng.doubles(cData.length, 0, height).toArray();
          for (int i = 0; i < cData.length; i++) {
  cData[i] = (float) cDataDbl[i];
          }
  c.put(0, 0, cData);
   
          // ------------------ Set up the non-linearly separable part of the training data ---------------
          // Generate random points for the classes 1 and 2
  trainClass = trainData.rowRange(nLinearSamples, 2 * NTRAINING_SAMPLES - nLinearSamples);
          // The x coordinate of the points is in [0.4, 0.6)
  c = trainClass.colRange(0, 1);
  cData = new float[(int) (c.total() * c.channels())];
  cDataDbl = rng.doubles(cData.length, 0.4 * width, 0.6 * width).toArray();
          for (int i = 0; i < cData.length; i++) {
  cData[i] = (float) cDataDbl[i];
          }
  c.put(0, 0, cData);
          // The y coordinate of the points is in [0, 1)
  c = trainClass.colRange(1, 2);
  cData = new float[(int) (c.total() * c.channels())];
  cDataDbl = rng.doubles(cData.length, 0, height).toArray();
          for (int i = 0; i < cData.length; i++) {
  cData[i] = (float) cDataDbl[i];
          }
  c.put(0, 0, cData);
   
          // ------------------------- Set up the labels for the classes---------------------------------
  labels.rowRange(0, NTRAINING_SAMPLES).setTo(new Scalar(1)); // Class 1
  labels.rowRange(NTRAINING_SAMPLES, 2 * NTRAINING_SAMPLES).setTo(new Scalar(2)); // Class 2
   
          // ------------------------ 2. Set up the support vector machines parameters--------------------
  System.out.println("Starting training process");
  SVM svm = SVM.create();
  svm.setType(SVM.C_SVC);
  svm.setC(0.1);
  svm.setKernel(SVM.LINEAR);
  svm.setTermCriteria(new TermCriteria(TermCriteria.MAX_ITER, (int) 1e7, 1e-6));
   
          // ------------------------ 3. Train the svm----------------------------------------------------
  svm.train(trainData, Ml.ROW_SAMPLE, labels);
  System.out.println("Finished training process");
   
          // ------------------------ 4. Show the decision regions----------------------------------------
          byte[] IData = new byte[(int) (I.total() * I.channels())];
  Mat sampleMat = new Mat(1, 2, CvType.CV_32F);
          float[] sampleMatData = new float[(int) (sampleMat.total() * sampleMat.channels())];
          for (int i = 0; i < I.rows(); i++) {
              for (int j = 0; j < I.cols(); j++) {
  sampleMatData[0] = j;
  sampleMatData[1] = i;
  sampleMat.put(0, 0, sampleMatData);
                  float response = svm.predict(sampleMat);
   
                  if (response == 1) {
  IData[(i * I.cols() + j) * I.channels()] = 0;
  IData[(i * I.cols() + j) * I.channels() + 1] = 100;
  IData[(i * I.cols() + j) * I.channels() + 2] = 0;
  } else if (response == 2) {
  IData[(i * I.cols() + j) * I.channels()] = 100;
  IData[(i * I.cols() + j) * I.channels() + 1] = 0;
  IData[(i * I.cols() + j) * I.channels() + 2] = 0;
                  }
              }
          }
  I.put(0, 0, IData);
   
          // ----------------------- 5. Show the training data--------------------------------------------
          int thick = -1;
          int lineType = Imgproc.LINE_8;
          float px, py;
          // Class 1
          float[] trainDataData = new float[(int) (trainData.total() * trainData.channels())];
  trainData.get(0, 0, trainDataData);
          for (int i = 0; i < NTRAINING_SAMPLES; i++) {
  px = trainDataData[i * trainData.cols()];
  py = trainDataData[i * trainData.cols() + 1];
  Imgproc.circle(I, new Point(px, py), 3, new Scalar(0, 255, 0), thick, lineType, 0);
          }
          // Class 2
          for (int i = NTRAINING_SAMPLES; i < 2 * NTRAINING_SAMPLES; ++i) {
  px = trainDataData[i * trainData.cols()];
  py = trainDataData[i * trainData.cols() + 1];
  Imgproc.circle(I, new Point(px, py), 3, new Scalar(255, 0, 0), thick, lineType, 0);
          }
   
          // ------------------------- 6. Show support vectors--------------------------------------------
  thick = 2;
  Mat sv = svm.getUncompressedSupportVectors();
          float[] svData = new float[(int) (sv.total() * sv.channels())];
  sv.get(0, 0, svData);
          for (int i = 0; i < sv.rows(); i++) {
  Imgproc.circle(I, new Point(svData[i * sv.cols()], svData[i * sv.cols() + 1]), 6, new Scalar(128, 128, 128),
  thick, lineType, 0);
          }
   
  Imgcodecs.imwrite("result.png", I); // save the Image
  HighGui.imshow("SVM for Non-Linear Training Data", I); // show it to the user
  HighGui.waitKey();
  System.exit(0);
      }
  }

Python

• 可下載程式碼：點選此處
• 程式碼速覽

  from __future__ import print_function
  import cv2 as cv
  import numpy as np
  import random as rng
   
  NTRAINING_SAMPLES = 100 # Number of training samples per class
  FRAC_LINEAR_SEP = 0.9 # Fraction of samples which compose the linear separable part
   
  # Data for visual representation
  WIDTH = 512
  HEIGHT = 512
  I = np.zeros((HEIGHT, WIDTH, 3), dtype=np.uint8)
   
  # --------------------- 1. Set up training data randomly ---------------------------------------
  trainData = np.empty((2*NTRAINING_SAMPLES, 2), dtype=np.float32)
  labels = np.empty((2*NTRAINING_SAMPLES, 1), dtype=np.int32)
   
  rng.seed(100) # Random value generation class
   
  # Set up the linearly separable part of the training data
  nLinearSamples = int(FRAC_LINEAR_SEP * NTRAINING_SAMPLES)
   
   
  trainClass = trainData[0:nLinearSamples,:]
  # The x coordinate of the points is in [0, 0.4)
  c = trainClass[:,0:1]
  c[:] = np.random.uniform(0.0, 0.4 * WIDTH, c.shape)
  # The y coordinate of the points is in [0, 1)
  c = trainClass[:,1:2]
  c[:] = np.random.uniform(0.0, HEIGHT, c.shape)
   
  # Generate random points for the class 2
  trainClass = trainData[2*NTRAINING_SAMPLES-nLinearSamples:2*NTRAINING_SAMPLES,:]
  # The x coordinate of the points is in [0.6, 1]
  c = trainClass[:,0:1]
  c[:] = np.random.uniform(0.6*WIDTH, WIDTH, c.shape)
  # The y coordinate of the points is in [0, 1)
  c = trainClass[:,1:2]
  c[:] = np.random.uniform(0.0, HEIGHT, c.shape)
   
   
  #------------------ Set up the non-linearly separable part of the training data ---------------
   
  trainClass = trainData[nLinearSamples:2*NTRAINING_SAMPLES-nLinearSamples,:]
  # The x coordinate of the points is in [0.4, 0.6)
  c = trainClass[:,0:1]
  c[:] = np.random.uniform(0.4*WIDTH, 0.6*WIDTH, c.shape)
  # The y coordinate of the points is in [0, 1)
  c = trainClass[:,1:2]
  c[:] = np.random.uniform(0.0, HEIGHT, c.shape)
   
   
  #------------------------- Set up the labels for the classes ---------------------------------
  labels[0:NTRAINING_SAMPLES,:] = 1 # Class 1
  labels[NTRAINING_SAMPLES:2*NTRAINING_SAMPLES,:] = 2 # Class 2
   
  #------------------------ 2. Set up the support vector machines parameters --------------------
  print('Starting training process')
   
  svm = cv.ml.SVM_create()
  svm.setType(cv.ml.SVM_C_SVC)
  svm.setC(0.1)
  svm.setKernel(cv.ml.SVM_LINEAR)
  svm.setTermCriteria((cv.TERM_CRITERIA_MAX_ITER, int(1e7), 1e-6))
   
   
  #------------------------ 3. Train the svm ----------------------------------------------------
   
  svm.train(trainData, cv.ml.ROW_SAMPLE, labels)
   
  print('Finished training process')
   
  #------------------------ 4. Show the decision regions ----------------------------------------
   
  green = (0,100,0)
  blue = (100,0,0)
  for i in range(I.shape[0])
      for j in range(I.shape[1])
  sampleMat = np.matrix([[j,i]], dtype=np.float32)
  response = svm.predict(sampleMat)[1]
   
          if response == 1
  I[i,j] = green
          elif response == 2
  I[i,j] = blue
   
   
  #----------------------- 5. Show the training data --------------------------------------------
   
  thick = -1
  # Class 1
  for i in range(NTRAINING_SAMPLES)
  px = trainData[i,0]
  py = trainData[i,1]
      cv.circle(I, (int(px), int(py)), 3, (0, 255, 0), thick)
   
  # Class 2
  for i in range(NTRAINING_SAMPLES, 2*NTRAINING_SAMPLES)
  px = trainData[i,0]
  py = trainData[i,1]
      cv.circle(I, (int(px), int(py)), 3, (255, 0, 0), thick)
   
   
  #------------------------- 6. Show support vectors --------------------------------------------
   
  thick = 2
  sv = svm.getUncompressedSupportVectors()
   
  for i in range(sv.shape[0])
      cv.circle(I, (int(sv[i,0]), int(sv[i,1])), 6, (128, 128, 128), thick)
   
   
  cv.imwrite('result.png', I) # save the Image
  cv.imshow('SVM for Non-Linear Training Data', I) # show it to the user
  cv.waitKey()

解釋

• 設定訓練資料

本練習的訓練資料由一組屬於兩個不同類別的帶標籤二維點組成。為了使練習更具吸引力，訓練資料是使用均勻機率密度函式（PDF）隨機生成的。

我們已將訓練資料的生成分為兩個主要部分。

在第一部分中，我們為兩個類別生成線性可分的資料。

C++

    // Generate random points for the class 1
    Mat trainClass = trainData.rowRange(0, nLinearSamples);
    // The x coordinate of the points is in [0, 0.4)
    Mat c = trainClass.colRange(0, 1);
rng.fill(c, RNG::UNIFORM, Scalar(0), Scalar(0.4 * WIDTH));
    // The y coordinate of the points is in [0, 1)
c = trainClass.colRange(1,2);
rng.fill(c, RNG::UNIFORM, Scalar(0), Scalar(HEIGHT));
 
    // Generate random points for the class 2
trainClass = trainData.rowRange(2*NTRAINING_SAMPLES-nLinearSamples, 2*NTRAINING_SAMPLES);
    // The x coordinate of the points is in [0.6, 1]
c = trainClass.colRange(0 , 1);
rng.fill(c, RNG::UNIFORM, Scalar(0.6*WIDTH), Scalar(WIDTH));
    // The y coordinate of the points is in [0, 1)
c = trainClass.colRange(1,2);
rng.fill(c, RNG::UNIFORM, Scalar(0), Scalar(HEIGHT));

Java

        // Generate random points for the class 1
Mat trainClass = trainData.rowRange(0, nLinearSamples);
        // The x coordinate of the points is in [0, 0.4)
Mat c = trainClass.colRange(0, 1);
        float[] cData = new float[(int) (c.total() * c.channels())];
        double[] cDataDbl = rng.doubles(cData.length, 0, 0.4f * width).toArray();
        for (int i = 0; i < cData.length; i++) {
cData[i] = (float) cDataDbl[i];
        }
c.put(0, 0, cData);
        // The y coordinate of the points is in [0, 1)
c = trainClass.colRange(1, 2);
cData = new float[(int) (c.total() * c.channels())];
cDataDbl = rng.doubles(cData.length, 0, height).toArray();
        for (int i = 0; i < cData.length; i++) {
cData[i] = (float) cDataDbl[i];
        }
c.put(0, 0, cData);
 
        // Generate random points for the class 2
trainClass = trainData.rowRange(2 * NTRAINING_SAMPLES - nLinearSamples, 2 * NTRAINING_SAMPLES);
        // The x coordinate of the points is in [0.6, 1]
c = trainClass.colRange(0, 1);
cData = new float[(int) (c.total() * c.channels())];
cDataDbl = rng.doubles(cData.length, 0.6 * width, width).toArray();
        for (int i = 0; i < cData.length; i++) {
cData[i] = (float) cDataDbl[i];
        }
c.put(0, 0, cData);
        // The y coordinate of the points is in [0, 1)
c = trainClass.colRange(1, 2);
cData = new float[(int) (c.total() * c.channels())];
cDataDbl = rng.doubles(cData.length, 0, height).toArray();
        for (int i = 0; i < cData.length; i++) {
cData[i] = (float) cDataDbl[i];
        }
c.put(0, 0, cData);

Python

trainClass = trainData[0:nLinearSamples,:]
# The x coordinate of the points is in [0, 0.4)
c = trainClass[:,0:1]
c[:] = np.random.uniform(0.0, 0.4 * WIDTH, c.shape)
# The y coordinate of the points is in [0, 1)
c = trainClass[:,1:2]
c[:] = np.random.uniform(0.0, HEIGHT, c.shape)
 
# Generate random points for the class 2
trainClass = trainData[2*NTRAINING_SAMPLES-nLinearSamples:2*NTRAINING_SAMPLES,:]
# The x coordinate of the points is in [0.6, 1]
c = trainClass[:,0:1]
c[:] = np.random.uniform(0.6*WIDTH, WIDTH, c.shape)
# The y coordinate of the points is in [0, 1)
c = trainClass[:,1:2]
c[:] = np.random.uniform(0.0, HEIGHT, c.shape)

在第二部分中，我們為兩個類別建立非線性可分的資料，即重疊資料。

C++

    // Generate random points for the classes 1 and 2
trainClass = trainData.rowRange(nLinearSamples, 2*NTRAINING_SAMPLES-nLinearSamples);
    // The x coordinate of the points is in [0.4, 0.6)
c = trainClass.colRange(0,1);
rng.fill(c, RNG::UNIFORM, Scalar(0.4*WIDTH), Scalar(0.6*WIDTH));
    // The y coordinate of the points is in [0, 1)
c = trainClass.colRange(1,2);
rng.fill(c, RNG::UNIFORM, Scalar(0), Scalar(HEIGHT));

Java

        // Generate random points for the classes 1 and 2
trainClass = trainData.rowRange(nLinearSamples, 2 * NTRAINING_SAMPLES - nLinearSamples);
        // The x coordinate of the points is in [0.4, 0.6)
c = trainClass.colRange(0, 1);
cData = new float[(int) (c.total() * c.channels())];
cDataDbl = rng.doubles(cData.length, 0.4 * width, 0.6 * width).toArray();
        for (int i = 0; i < cData.length; i++) {
cData[i] = (float) cDataDbl[i];
        }
c.put(0, 0, cData);
        // The y coordinate of the points is in [0, 1)
c = trainClass.colRange(1, 2);
cData = new float[(int) (c.total() * c.channels())];
cDataDbl = rng.doubles(cData.length, 0, height).toArray();
        for (int i = 0; i < cData.length; i++) {
cData[i] = (float) cDataDbl[i];
        }
c.put(0, 0, cData);

Python

trainClass = trainData[nLinearSamples:2*NTRAINING_SAMPLES-nLinearSamples,:]
# The x coordinate of the points is in [0.4, 0.6)
c = trainClass[:,0:1]
c[:] = np.random.uniform(0.4*WIDTH, 0.6*WIDTH, c.shape)
# The y coordinate of the points is in [0, 1)
c = trainClass[:,1:2]
c[:] = np.random.uniform(0.0, HEIGHT, c.shape)

• 設定SVM引數

注意

在上一教程支援向量機簡介中，解釋了我們在訓練SVM之前在此處配置的cv::ml::SVM類的屬性。

C++

    Ptr<SVM> svm = SVM::create();
svm->setType(SVM::C_SVC);
svm->setC(0.1);
svm->setKernel(SVM::LINEAR);
svm->setTermCriteria(TermCriteria(TermCriteria::MAX_ITER, (int)1e7, 1e-6));

Java

SVM svm = SVM.create();
svm.setType(SVM.C_SVC);
svm.setC(0.1);
svm.setKernel(SVM.LINEAR);
svm.setTermCriteria(new TermCriteria(TermCriteria.MAX_ITER, (int) 1e7, 1e-6));

Python

svm = cv.ml.SVM_create()
svm.setType(cv.ml.SVM_C_SVC)
svm.setC(0.1)
svm.setKernel(cv.ml.SVM_LINEAR)
svm.setTermCriteria((cv.TERM_CRITERIA_MAX_ITER, int(1e7), 1e-6))

我們在此處的配置與作為參考的上一教程（支援向量機簡介）中的配置僅有兩個不同之處。

• C。我們在此處選擇了一個較小的引數值，以便在最佳化中不過多懲罰誤分類錯誤。這樣做的目的是為了獲得一個接近直觀預期結果的解決方案。但是，我們建議透過調整此引數來更深入地瞭解問題。

  注意

  在這種情況下，重疊區域中只有很少的點。透過將FRAC_LINEAR_SEP設定為較小的值，可以增加點的密度，從而深入探討引數C的影響。

• 演算法的終止準則。為了正確解決非線性可分離訓練資料問題，必須大幅增加最大迭代次數。特別是，我們將此值增加了五個數量級。
• 訓練SVM

我們呼叫cv::ml::SVM::train方法來構建SVM模型。請注意，訓練過程可能需要相當長的時間。執行程式時請耐心等待。

C++

svm->train(trainData, ROW_SAMPLE, labels);

Java

svm.train(trainData, Ml.ROW_SAMPLE, labels);

Python

svm.train(trainData, cv.ml.ROW_SAMPLE, labels)

• 顯示決策區域

cv::ml::SVM::predict方法用於使用訓練好的SVM對輸入樣本進行分類。在此示例中，我們使用此方法根據SVM的預測為空間著色。換句話說，遍歷影像，將其畫素解釋為笛卡爾平面上的點。每個點根據SVM預測的類別著色；如果是標籤為1的類別，則為深綠色；如果是標籤為2的類別，則為深藍色。

C++

    Vec3b green(0,100,0), blue(100,0,0);
    for (int i = 0; i < I.rows; i++)
    {
        for (int j = 0; j < I.cols; j++)
        {
            Mat sampleMat = (Mat_<float>(1,2) << j, i);
            float response = svm->predict(sampleMat);
 
            if (response == 1) I.at<Vec3b>(i,j) = green;
            else if (response == 2) I.at<Vec3b>(i,j) = blue;
        }
    }

Java

        byte[] IData = new byte[(int) (I.total() * I.channels())];
Mat sampleMat = new Mat(1, 2, CvType.CV_32F);
        float[] sampleMatData = new float[(int) (sampleMat.total() * sampleMat.channels())];
        for (int i = 0; i < I.rows(); i++) {
            for (int j = 0; j < I.cols(); j++) {
sampleMatData[0] = j;
sampleMatData[1] = i;
sampleMat.put(0, 0, sampleMatData);
                float response = svm.predict(sampleMat);
 
                if (response == 1) {
IData[(i * I.cols() + j) * I.channels()] = 0;
IData[(i * I.cols() + j) * I.channels() + 1] = 100;
IData[(i * I.cols() + j) * I.channels() + 2] = 0;
} else if (response == 2) {
IData[(i * I.cols() + j) * I.channels()] = 100;
IData[(i * I.cols() + j) * I.channels() + 1] = 0;
IData[(i * I.cols() + j) * I.channels() + 2] = 0;
                }
            }
        }
I.put(0, 0, IData);

Python

green = (0,100,0)
blue = (100,0,0)
for i in range(I.shape[0])
    for j in range(I.shape[1])
sampleMat = np.matrix([[j,i]], dtype=np.float32)
response = svm.predict(sampleMat)[1]
 
        if response == 1
I[i,j] = green
        elif response == 2
I[i,j] = blue

• 顯示訓練資料

cv::circle方法用於顯示組成訓練資料的樣本。標籤為1的類別的樣本顯示為淺綠色，標籤為2的類別的樣本顯示為淺藍色。

C++

    int thick = -1;
    float px, py;
    // Class 1
    for (int i = 0; i < NTRAINING_SAMPLES; i++)
    {
px = trainData.at<float>(i,0);
py = trainData.at<float>(i,1);
        circle(I, Point( (int) px, (int) py ), 3, Scalar(0, 255, 0), thick);
    }
    // Class 2
    for (int i = NTRAINING_SAMPLES; i <2*NTRAINING_SAMPLES; i++)
    {
px = trainData.at<float>(i,0);
py = trainData.at<float>(i,1);
        circle(I, Point( (int) px, (int) py ), 3, Scalar(255, 0, 0), thick);
    }

Java

        int thick = -1;
        int lineType = Imgproc.LINE_8;
        float px, py;
        // Class 1
        float[] trainDataData = new float[(int) (trainData.total() * trainData.channels())];
trainData.get(0, 0, trainDataData);
        for (int i = 0; i < NTRAINING_SAMPLES; i++) {
px = trainDataData[i * trainData.cols()];
py = trainDataData[i * trainData.cols() + 1];
Imgproc.circle(I, new Point(px, py), 3, new Scalar(0, 255, 0), thick, lineType, 0);
        }
        // Class 2
        for (int i = NTRAINING_SAMPLES; i < 2 * NTRAINING_SAMPLES; ++i) {
px = trainDataData[i * trainData.cols()];
py = trainDataData[i * trainData.cols() + 1];
Imgproc.circle(I, new Point(px, py), 3, new Scalar(255, 0, 0), thick, lineType, 0);
        }

Python

thick = -1
# Class 1
for i in range(NTRAINING_SAMPLES)
px = trainData[i,0]
py = trainData[i,1]
    cv.circle(I, (int(px), int(py)), 3, (0, 255, 0), thick)
 
# Class 2
for i in range(NTRAINING_SAMPLES, 2*NTRAINING_SAMPLES)
px = trainData[i,0]
py = trainData[i,1]
    cv.circle(I, (int(px), int(py)), 3, (255, 0, 0), thick)

• 支援向量

我們在此處使用一些方法來獲取有關支援向量的資訊。cv::ml::SVM::getSupportVectors方法獲取所有支援向量。我們在這裡使用此方法來查詢作為支援向量的訓練示例並對其進行突出顯示。

C++

thick = 2;
    Mat sv = svm->getUncompressedSupportVectors();
 
    for (int i = 0; i < sv.rows; i++)
    {
        const float* v = sv.ptr<float>(i);
        circle(I, Point( (int) v[0], (int) v[1]), 6, Scalar(128, 128, 128), thick);
    }

Java

thick = 2;
Mat sv = svm.getUncompressedSupportVectors();
        float[] svData = new float[(int) (sv.total() * sv.channels())];
sv.get(0, 0, svData);
        for (int i = 0; i < sv.rows(); i++) {
Imgproc.circle(I, new Point(svData[i * sv.cols()], svData[i * sv.cols() + 1]), 6, new Scalar(128, 128, 128),
thick, lineType, 0);
        }

Python

thick = 2
sv = svm.getUncompressedSupportVectors()
 
for i in range(sv.shape[0])
    cv.circle(I, (int(sv[i,0]), int(sv[i,1])), 6, (128, 128, 128), thick)

結果

• 程式碼開啟一張影像並顯示兩個類別的訓練示例。一個類別的點用淺綠色表示，另一個類別用淺藍色表示。
• SVM經過訓練並用於分類影像的所有畫素。這導致影像被劃分為藍色區域和綠色區域。兩個區域之間的邊界是分離超平面。由於訓練資料是非線性可分的，可以看到兩個類別中的一些示例被錯誤分類；一些綠點落在藍色區域，一些藍點落在綠色區域。
• 最後，使用訓練示例周圍的灰色環顯示支援向量。

你可以在此處觀看其執行時例項。