this is a simple Qt+OpenCV+OpenGL integration. this entry will show you how you can capture webcam image using opencv and draw it in Qt apps using opengl texture on GLWidget. requires Qt & OpenCV 2.1 here is our main window code for query and process image from webcam query image from camera
void Opencv2Qt::processCam() {
if (this->capture.isOpened()) {
timer.restart();
Mat frame;
capture >> frame;
if (frame.data) {
this->processFrame(frame);
glWidget->sendImage(&frame);
ui->statusBar->showMessage("Running....");
QTimer::singleShot(25, this, SLOT(processCam()));
}
}
return;
}
lets do something with the frame, this is simple example how to change brightness and contrast
void Opencv2Qt::processFrame(Mat& img) {
const double brightness_gain = 0.5;
const double contrast_gain = 0.6;
Mat white(img.size(), CV_8UC3);
white=Scalar(255,255,255);
addWeighted(img, contrast_gain, white, 1,-128 + brightness_gain, img);
}
i used QTimer to query for new image every 20 milisecond, this is approximately 25-30 fps. enough for conventional webcam. more on how to draw our image to Qt after the jump this is the function to recieve cv::Mat image and convert to opengl image format we create a class for GLWidget
void GLWidget::sendImage(Mat* img) {
qframe = QImage((const unsigned char*)(img->data), img->cols, img->rows, img->step,
QImage::Format_RGB888).rgbSwapped();
qframe = QGLWidget::convertToGLFormat(qframe);
this->updateGL();
}
now we will paint our image using PaintGL() function in this example i use 2D texture to render the image. alternatively you can use glDrawPixels to draw the image.
void GLWidget::paintGL() {
glClear (GL_COLOR_BUFFER_BIT);
glClearColor (0.0,0.0,0.0,1.0);
if (!qframe.isNull()) {
qframe = qframe.scaled(this->size(), Qt::IgnoreAspectRatio,Qt::SmoothTransformation);
// you can use glDrawPixels directly
// glDrawPixels(qframe.width(),qframe.height(), GL_RGBA, GL_UNSIGNED_BYTE, qframe.bits());
// or do 2D texture mapping
glDisable(GL_DEPTH_TEST);
glMatrixMode(GL_PROJECTION);
glLoadIdentity();
gluOrtho2D(0,qframe.width(),qframe.height(),0);
glMatrixMode(GL_MODELVIEW);
glLoadIdentity();
glEnable(GL_TEXTURE_2D);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_NEAREST);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_NEAREST);
glTexImage2D( GL_TEXTURE_2D, 0, 4, qframe.width(), qframe.height(), 0, \
GL_RGBA, GL_UNSIGNED_BYTE, qframe.bits() );
glBegin(GL_QUADS);
glTexCoord2f(0,0); glVertex2f(0,qframe.height());
glTexCoord2f(0,1); glVertex2f(0,0);
glTexCoord2f(1,1); glVertex2f(qframe.width(),0);
glTexCoord2f(1,0); glVertex2f(qframe.width(),qframe.height());
glEnd();
glDisable(GL_TEXTURE_2D);
// .... end
// some example of alpha blending
//glEnable(GL_DEPTH_TEST);
//glEnable(GL_BLEND);
//glBlendFunc(GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA);
//glColor4f(0.0f,1.0f,1.0f, 0.9f);
glFlush();
}
// qDebug() << "Drawing...";
}
...that's it, all done!
footnote:
1. why use PaintGL instead of QPainter? oh well..the answer is simple it's fast and can be utilize for complex image processing rendering such as 3D augmented reality.
2. note that this is direct implementation of image capturing from webcam using QTimer. for heavy processing job you might want to consider running processsCAM() and processingFrame() job on different thread. if i have time, i'll show how to do it in next entry 3. you may download opencv2Qt source code for details until then, happy coding and hope this code can be useful to someone... :)
Brilliant post, thank you very much. This is exactly what I was looking for. Now I am going to try and modify the code for my image processing tasks :)
I try to use this on Mac OS X (installed OpenCV 2.2.0 via MacPorts) but I get a linker error:
Undefined symbols:
"cv::Ptr::delete_obj()", referenced from:
Opencv2Qt::startCam() in opencv2qt.o
ld: symbol(s) not found
Any ideas / help would be appreciated.
Looks like that there were some changes from 2.1.0 to 2.2.0. Tested it with 2.1.0 (Linux) worked fine. With 2.2.0 (Linux and Mac): Error. :-(
Since I am not the OpenCV expert, any idea?
Thanks Markus.
please make sure the linker and path to opencv library is correct and edit your opencv2Qt.pro file accordingly. take a look at OpenCV-2.2.0/unix-install/opencv.pc it will give you some idea =)
Undefined symbols:
“cv::Ptr::delete_obj()”, referenced from:
Opencv2Qt::startCam() in opencv2qt.o
ld: symbol(s) not found
opencv 2.2.0 bug
solution
https://code.ros.org/trac/opencv/changeset/4313/branches/2.2/opencv#file0
Thanks, brilliant post.
I personally prefer your solution because it is small, direct and simple. I did want to mention a similar (and much larger) project that supports Qt and DirectShow called VideoMan. If this class is not enough, check out that project. Great work!
This post is so brilliant.
I try to use this under Windows 7 (installed OpenCV 2.2.0),
but I get two error when it is building on Qt,
1. cannot find -lml
2. collect2: ld returned 1 exit status
Any ideas / help would be appreciated.
Very nice, thx!
But I have a Question:
I implemented cvtColor and Canny and created 2 more glwidgets. 1 show the original image, 2nd the grayscale and 3rd show the edge image.
Problem was that "sendImage(Mat* img)" expects a "QImage::Format_RGB888". The resulting output was a black screen on both, grayscale and edge images.
So I cloned this function and called it sendImageGray testwise and changed FORMAT_RGB888 to FORMAT_Mono and removed ".rgbSwapped()".
Now it's working as it should showing original, grayscale and edge images. But one problem remains:
The grayscale image looks somehow messed up, lots of striped and so on and it has some kind of zoom factor in it. looks interpolated or smthg.
The resulting edge image is correct but also has this zoom factor so I think it's just a problem with viewing the processed images, img-data seems correct.
Any Ideas on which format I should use or how to set other parameters for displaying grayscale correct? What is "img->step" for?
Anyways thx for this really great Tutorial. I'm very interested in OpenCV and I love QT and the OpenGL Boost, so this is best:P
keep up the good work!
im a java programmer, but i think building web sites only for life do sucks
and play my game , comments are welcome!
http://web.tiscali.it/claudiopeppe
Hi, thanks very much for this post. I can imagine it'll help quite a lot of people. I noticed you're using the .rgbSwapped() function. It might be better to let OpenGL (and hopefully the hardware) do the swapping, instead of Qt with .rgbSwapped(). This can be done in glTexImage2D(). You have to use GL_RGB (instead of 4) for internalFormat, and GL_BGR for the pixel format. For function reference, see http://www.opengl.org/sdk/docs/man/xhtml/glTexImage2D.xml
Those people here did the same: http://www.rauwendaal.net/blog/opencvandopengl-1 and http://blog.damiles.com/?p=9
Combine everything with your solution, and you get the best performance :-)
Thanks again,
Linus Atorf
@S3dr1ck, the “img->step” is a property of the OpenCV matrix data structure. If you've got pixel data in a regular old-school array, it will be 0. But if you pad additional bytes for better performance (i.e. alignment, so 24bit will actually be stored as 32bit), you have to skip this data. The same happens when you select an ROI (region of interest) aka submatrix from the image. Then your data is not continous in memory anymore, and you have to add this step to your loop. See http://opencv.willowgarage.com/documentation/cpp/core_basic_structures.html#mat for more info on this.
i'm glad to see it can help peoples and thanks Linus Atorf for that additional informations
Can any one of you give a step-by-step instructions how to include the source codes in a project and run them !
I use Visual Studio 2008 with Qt-Add on...but I don't know what kind of project to create from the options..
thank you.