Video Manipulation with the Canvas Element

Video Manipulation with the Canvas Element

One of the designers in my company asked me to help him with the creation of a transparent video which runs on top of an image in HTML. So I did some digging and found a very good resource to do that - Manipulating video using canvas. This post will explain how to combine the video and canvas elements in order to create transparency inside a video during runtime using JavaScript. I’ve based the code on the previous link so they deserve all the credit. You can download the demo code from here.

The Demands

The designer has created a simple movie file with a red square that is floating on top of a black background. He wanted that during runtime, the black background will be transparent so the image that he wants to show underneath will be visible. Here is a figure that shows how the movie looks like:

MoviePicture

The HTML File

<!DOCTYPE HTML>
<html>
<head>
    <meta http-equiv="Content-Type" content="text/html; charset=utf-8">
    <title>Working with Canvas</title>
    <script type="text/javascript" src="main.js"></script>
    <link href="style.css" rel="stylesheet" />
</head>
<body onload="handlePageLoad()">
    <video id="video" controls="controls" width="997px" height="587px" autoplay>
        <source src="vid.mp4" type="video/mp4" />
    </video>
    <div>
        <canvas id="c1" width="997px" height="587px" />
        <canvas id="c2" width="997px" height="587px" />
    </div>
</body>
</html>

The HTML includes two canvas elements and the video element. I use two canvas elements because one will hold the buffer that I’m working on (and make the black background transparent) and the other will show the movie after the adjustments are finished. I also use the following style.css file: 

#c1
{
    background-image: url(Bg.png);
    background-repeat: no-repeat;
}
#c2
{
    display: none;
}
#video
{
    display: none;
}

In the CSS, I’m making sure not to display the buffer canvas and the video element and to show the background image in the first canvas element.

The Real Thing – The JavaScript Code

Here is the implementation of the JavaScript file that will help you do the job: 

var video;
var ctx1;
var ctx2;
var width;
var height;

function timerCallback() {
    if (video.paused || video.ended) {
        return;
    }
    generateFrame();
    setTimeout(function () {
        timerCallback();
    }, 0);
}

function handlePageLoad() {
    video = document.getElementById("video");
    var c1 = document.getElementById("c1");
    ctx1 = this.c1.getContext("2d");
    var c2 = document.getElementById("c2");
    ctx2 = this.c2.getContext("2d");
    video.addEventListener("play", function () {
        width = this.videoWidth;
        height = this.videoHeight;
        timerCallback();
    }, false);
}

function generateFrame() {
    ctx2.drawImage(video, 0, 0, width, height);
    var frame = ctx2.getImageData(0, 0, width, height);
    var len = frame.data.length / 4;

    for (var i = 0; i < len; i++) {
        var r = frame.data[i * 4 + 0];
        var g = frame.data[i * 4 + 1];
        var b = frame.data[i * 4 + 2];
        if (g < 10 && r < 10 && b < 10) {
            frame.data[i * 4 + 3] = 0;
        }
    }
    ctx1.putImageData(frame, 0, 0);
    return;
}

Let's deep dive into the things that the functions are doing:

  • The handlePageLoad function is responsible to initialize all the canvas contexts and to add a listener to the play event of the video. When it occurs, it will start the timerCallback function.
  • The timerCallback function is responsible to stop the generation of frames when the video stops and to generate the current frame using the generateFrame function.
  • The generateFrame function is the function that does the heavy lifting of drawing the transparent video on the canvas elements. It draws the video frame to the buffer canvas and then extracts that frame. After extracting the frame, it passes on every pixel and checks whether it is black. If so, it sets its opacity to transparent.

The effect will look like:

TheFullExample

Summary

In the post, I showed you how to use a canvas in order to make changes to a running video on the fly. This is only an experiment that I made to show that it is possible to do that. The performance of this implementation isn’t so good and I would have thought about other solutions such as a video that includes the background image and the movie.

关键字 video,manipulation,with,the,canvas,element
推荐.NET配套的通用数据层ORM框架：CYQ.Data 通用数据层框架
新浪微博粉丝精灵，刷粉丝、刷评论、刷转发、企业商家微博营销必备工具"