Table of Contents
Prev Tutorial: Detection of planar objects
Next Tutorial: AKAZE and ORB planar tracking
Original author | Fedor Morozov |
Compatibility | OpenCV >= 3.0 |
Introduction
In this tutorial we will learn how to use AKAZE [ANB13] local features to detect and match keypoints on two images. We will find keypoints on a pair of images with given homography matrix, match them and count the number of inliers (i.e. matches that fit in the given homography).
You can find expanded version of this example here: https://github.com/pablofdezalc/test_kaze_akaze_opencv
Data
We are going to use images 1 and 3 from Graffiti sequence of Oxford dataset.

Homography is given by a 3 by 3 matrix:
You can find the images (graf1.png, graf3.png) and homography (H1to3p.xml) in opencv/samples/data/.
Source Code
- Downloadable code: Click here
- Code at glance: #include <opencv2/features2d.hpp>#include <opencv2/imgproc.hpp>#include <opencv2/highgui.hpp>#include <iostream>using namespace std;using namespace cv;const float inlier_threshold = 2.5f; // Distance threshold to identify inliers with homography checkconst float nn_match_ratio = 0.8f; // Nearest neighbor matching ratioint main(int argc, char* argv[]){CommandLineParser parser(argc, argv,"{@img1 | graf1.png | input image 1}""{@img2 | graf3.png | input image 2}""{@homography | H1to3p.xml | homography matrix}");Mat homography;fs.getFirstTopLevelNode() >> homography;vector<KeyPoint> kpts1, kpts2;Mat desc1, desc2;Ptr<AKAZE> akaze = AKAZE::create();akaze->detectAndCompute(img1, noArray(), kpts1, desc1);akaze->detectAndCompute(img2, noArray(), kpts2, desc2);BFMatcher matcher(NORM_HAMMING);vector< vector<DMatch> > nn_matches;matcher.knnMatch(desc1, desc2, nn_matches, 2);vector<KeyPoint> matched1, matched2;for(size_t i = 0; i < nn_matches.size(); i++) {DMatch first = nn_matches[i][0];float dist1 = nn_matches[i][0].distance;float dist2 = nn_matches[i][1].distance;if(dist1 < nn_match_ratio * dist2) {matched1.push_back(kpts1[first.queryIdx]);matched2.push_back(kpts2[first.trainIdx]);}}vector<DMatch> good_matches;vector<KeyPoint> inliers1, inliers2;for(size_t i = 0; i < matched1.size(); i++) {col.at<double>(0) = matched1[i].pt.x;col.at<double>(1) = matched1[i].pt.y;col = homography * col;col /= col.at<double>(2);pow(col.at<double>(1) - matched2[i].pt.y, 2));if(dist < inlier_threshold) {int new_i = static_cast<int>(inliers1.size());inliers1.push_back(matched1[i]);inliers2.push_back(matched2[i]);good_matches.push_back(DMatch(new_i, new_i, 0));}}Mat res;drawMatches(img1, inliers1, img2, inliers2, good_matches, res);imwrite("akaze_result.png", res);double inlier_ratio = inliers1.size() / (double) matched1.size();cout << "A-KAZE Matching Results" << endl;cout << "*******************************" << endl;cout << "# Keypoints 1: \t" << kpts1.size() << endl;cout << "# Keypoints 2: \t" << kpts2.size() << endl;cout << "# Matches: \t" << matched1.size() << endl;cout << "# Inliers: \t" << inliers1.size() << endl;cout << "# Inliers Ratio: \t" << inlier_ratio << endl;cout << endl;imshow("result", res);waitKey();return 0;}XML/YAML/JSON file storage class that encapsulates all the information necessary for writing or readi...Definition: persistence.hpp:304void drawMatches(InputArray img1, const std::vector< KeyPoint > &keypoints1, InputArray img2, const std::vector< KeyPoint > &keypoints2, const std::vector< DMatch > &matches1to2, InputOutputArray outImg, const Scalar &matchColor=Scalar::all(-1), const Scalar &singlePointColor=Scalar::all(-1), const std::vector< char > &matchesMask=std::vector< char >(), DrawMatchesFlags flags=DrawMatchesFlags::DEFAULT)Draws the found matches of keypoints from two images.void imshow(const String &winname, InputArray mat)Displays an image in the specified window.CV_EXPORTS_W bool imwrite(const String &filename, InputArray img, const std::vector< int > ¶ms=std::vector< int >())Saves an image to a specified file."black box" representation of the file storage associated with a file on disk.Definition: core.hpp:106STL namespace.
- Downloadable code: Click here
- Code at glance: import java.io.File;import java.io.IOException;import java.util.ArrayList;import java.util.List;import javax.xml.parsers.DocumentBuilder;import javax.xml.parsers.DocumentBuilderFactory;import javax.xml.parsers.ParserConfigurationException;import org.opencv.core.Core;import org.opencv.core.CvType;import org.opencv.core.DMatch;import org.opencv.core.KeyPoint;import org.opencv.core.Mat;import org.opencv.core.MatOfDMatch;import org.opencv.core.MatOfKeyPoint;import org.opencv.core.Scalar;import org.opencv.features2d.AKAZE;import org.opencv.features2d.DescriptorMatcher;import org.opencv.features2d.Features2d;import org.opencv.highgui.HighGui;import org.opencv.imgcodecs.Imgcodecs;import org.w3c.dom.Document;import org.xml.sax.SAXException;class AKAZEMatch {public void run(String[] args) {String filename1 = args.length > 2 ? args[0] : "../data/graf1.png";String filename2 = args.length > 2 ? args[1] : "../data/graf3.png";String filename3 = args.length > 2 ? args[2] : "../data/H1to3p.xml";Mat img1 = Imgcodecs.imread(filename1, Imgcodecs.IMREAD_GRAYSCALE);Mat img2 = Imgcodecs.imread(filename2, Imgcodecs.IMREAD_GRAYSCALE);if (img1.empty() || img2.empty()) {System.err.println("Cannot read images!");System.exit(0);}File file = new File(filename3);DocumentBuilderFactory documentBuilderFactory = DocumentBuilderFactory.newInstance();DocumentBuilder documentBuilder;Document document;Mat homography = new Mat(3, 3, CvType.CV_64F);double[] homographyData = new double[(int) (homography.total()*homography.channels())];try {documentBuilder = documentBuilderFactory.newDocumentBuilder();document = documentBuilder.parse(file);String homographyStr = document.getElementsByTagName("data").item(0).getTextContent();String[] splited = homographyStr.split("\\s+");int idx = 0;for (String s : splited) {if (!s.isEmpty()) {homographyData[idx] = Double.parseDouble(s);idx++;}}} catch (ParserConfigurationException e) {e.printStackTrace();System.exit(0);} catch (SAXException e) {e.printStackTrace();System.exit(0);} catch (IOException e) {e.printStackTrace();System.exit(0);}homography.put(0, 0, homographyData);AKAZE akaze = AKAZE.create();MatOfKeyPoint kpts1 = new MatOfKeyPoint(), kpts2 = new MatOfKeyPoint();Mat desc1 = new Mat(), desc2 = new Mat();akaze.detectAndCompute(img1, new Mat(), kpts1, desc1);akaze.detectAndCompute(img2, new Mat(), kpts2, desc2);DescriptorMatcher matcher = DescriptorMatcher.create(DescriptorMatcher.BRUTEFORCE_HAMMING);List<MatOfDMatch> knnMatches = new ArrayList<>();matcher.knnMatch(desc1, desc2, knnMatches, 2);float ratioThreshold = 0.8f; // Nearest neighbor matching ratioList<KeyPoint> listOfMatched1 = new ArrayList<>();List<KeyPoint> listOfMatched2 = new ArrayList<>();List<KeyPoint> listOfKeypoints1 = kpts1.toList();List<KeyPoint> listOfKeypoints2 = kpts2.toList();for (int i = 0; i < knnMatches.size(); i++) {DMatch[] matches = knnMatches.get(i).toArray();float dist1 = matches[0].distance;float dist2 = matches[1].distance;if (dist1 < ratioThreshold * dist2) {listOfMatched1.add(listOfKeypoints1.get(matches[0].queryIdx));listOfMatched2.add(listOfKeypoints2.get(matches[0].trainIdx));}}double inlierThreshold = 2.5; // Distance threshold to identify inliers with homography checkList<KeyPoint> listOfInliers1 = new ArrayList<>();List<KeyPoint> listOfInliers2 = new ArrayList<>();List<DMatch> listOfGoodMatches = new ArrayList<>();for (int i = 0; i < listOfMatched1.size(); i++) {Mat col = new Mat(3, 1, CvType.CV_64F);double[] colData = new double[(int) (col.total() * col.channels())];colData[0] = listOfMatched1.get(i).pt.x;colData[1] = listOfMatched1.get(i).pt.y;colData[2] = 1.0;col.put(0, 0, colData);Mat colRes = new Mat();Core.gemm(homography, col, 1.0, new Mat(), 0.0, colRes);colRes.get(0, 0, colData);Core.multiply(colRes, new Scalar(1.0 / colData[2]), col);col.get(0, 0, colData);double dist = Math.sqrt(Math.pow(colData[0] - listOfMatched2.get(i).pt.x, 2) +Math.pow(colData[1] - listOfMatched2.get(i).pt.y, 2));if (dist < inlierThreshold) {listOfGoodMatches.add(new DMatch(listOfInliers1.size(), listOfInliers2.size(), 0));listOfInliers1.add(listOfMatched1.get(i));listOfInliers2.add(listOfMatched2.get(i));}}Mat res = new Mat();MatOfKeyPoint inliers1 = new MatOfKeyPoint(listOfInliers1.toArray(new KeyPoint[listOfInliers1.size()]));MatOfKeyPoint inliers2 = new MatOfKeyPoint(listOfInliers2.toArray(new KeyPoint[listOfInliers2.size()]));MatOfDMatch goodMatches = new MatOfDMatch(listOfGoodMatches.toArray(new DMatch[listOfGoodMatches.size()]));Features2d.drawMatches(img1, inliers1, img2, inliers2, goodMatches, res);Imgcodecs.imwrite("akaze_result.png", res);double inlierRatio = listOfInliers1.size() / (double) listOfMatched1.size();System.out.println("A-KAZE Matching Results");System.out.println("*******************************");System.out.println("# Keypoints 1: \t" + listOfKeypoints1.size());System.out.println("# Keypoints 2: \t" + listOfKeypoints2.size());System.out.println("# Matches: \t" + listOfMatched1.size());System.out.println("# Inliers: \t" + listOfInliers1.size());System.out.println("# Inliers Ratio: \t" + inlierRatio);HighGui.imshow("result", res);HighGui.waitKey();System.exit(0);}}public class AKAZEMatchDemo {public static void main(String[] args) {// Load the native OpenCV librarySystem.loadLibrary(Core.NATIVE_LIBRARY_NAME);new AKAZEMatch().run(args);}}
- Downloadable code: Click here
- Code at glance: from __future__ import print_functionimport cv2 as cvimport numpy as npimport argparsefrom math import sqrtparser = argparse.ArgumentParser(description='Code for AKAZE local features matching tutorial.')parser.add_argument('--input1', help='Path to input image 1.', default='graf1.png')parser.add_argument('--input2', help='Path to input image 2.', default='graf3.png')parser.add_argument('--homography', help='Path to the homography matrix.', default='H1to3p.xml')args = parser.parse_args()img1 = cv.imread(cv.samples.findFile(args.input1), cv.IMREAD_GRAYSCALE)img2 = cv.imread(cv.samples.findFile(args.input2), cv.IMREAD_GRAYSCALE)if img1 is None or img2 is None:print('Could not open or find the images!')exit(0)fs = cv.FileStorage(cv.samples.findFile(args.homography), cv.FILE_STORAGE_READ)homography = fs.getFirstTopLevelNode().mat()akaze = cv.AKAZE_create()kpts1, desc1 = akaze.detectAndCompute(img1, None)kpts2, desc2 = akaze.detectAndCompute(img2, None)matcher = cv.DescriptorMatcher_create(cv.DescriptorMatcher_BRUTEFORCE_HAMMING)nn_matches = matcher.knnMatch(desc1, desc2, 2)matched1 = []matched2 = []nn_match_ratio = 0.8 # Nearest neighbor matching ratiofor m, n in nn_matches:if m.distance < nn_match_ratio * n.distance:matched1.append(kpts1[m.queryIdx])matched2.append(kpts2[m.trainIdx])inliers1 = []inliers2 = []good_matches = []inlier_threshold = 2.5 # Distance threshold to identify inliers with homography checkfor i, m in enumerate(matched1):col = np.ones((3,1), dtype=np.float64)col[0:2,0] = m.ptcol = np.dot(homography, col)col /= col[2,0]dist = sqrt(pow(col[0,0] - matched2[i].pt[0], 2) +\pow(col[1,0] - matched2[i].pt[1], 2))if dist < inlier_threshold:good_matches.append(cv.DMatch(len(inliers1), len(inliers2), 0))inliers1.append(matched1[i])inliers2.append(matched2[i])res = np.empty((max(img1.shape[0], img2.shape[0]), img1.shape[1]+img2.shape[1], 3), dtype=np.uint8)cv.drawMatches(img1, inliers1, img2, inliers2, good_matches, res)cv.imwrite("akaze_result.png", res)inlier_ratio = len(inliers1) / float(len(matched1))print('A-KAZE Matching Results')print('*******************************')print('# Keypoints 1: \t', len(kpts1))print('# Keypoints 2: \t', len(kpts2))print('# Matches: \t', len(matched1))print('# Inliers: \t', len(inliers1))print('# Inliers Ratio: \t', inlier_ratio)cv.imshow('result', res)cv::String findFile(const cv::String &relative_path, bool required=true, bool silentMode=false)Try to find requested data file.CV_EXPORTS_W Mat imread(const String &filename, int flags=IMREAD_COLOR)Loads an image from a file.
Explanation
- Load images and homography
We are loading grayscale images here. Homography is stored in the xml created with FileStorage.
- Detect keypoints and compute descriptors using AKAZE
We create AKAZE and detect and compute AKAZE keypoints and descriptors. Since we don't need the mask parameter, noArray() is used.
- Use brute-force matcher to find 2-nn matches
We use Hamming distance, because AKAZE uses binary descriptor by default.
- Use 2-nn matches and ratio criterion to find correct keypoint matches C++vector<KeyPoint> matched1, matched2;for(size_t i = 0; i < nn_matches.size(); i++) {DMatch first = nn_matches[i][0];float dist1 = nn_matches[i][0].distance;float dist2 = nn_matches[i][1].distance;if(dist1 < nn_match_ratio * dist2) {matched1.push_back(kpts1[first.queryIdx]);matched2.push_back(kpts2[first.trainIdx]);}}
If the closest match distance is significantly lower than the second closest one, then the match is correct (match is not ambiguous).
- Check if our matches fit in the homography model
If the distance from first keypoint's projection to the second keypoint is less than threshold, then it fits the homography model.
We create a new set of matches for the inliers, because it is required by the drawing function.
- Output results
Here we save the resulting image and print some statistics.
Results
Found matches

Depending on your OpenCV version, you should get results coherent with: