NutUI video component development experience

Posted Jun 5, 202014 min read

Introduction

When it comes to introducing a video in the project, we will definitely think of the Video tag provided by HTML5, which provides us with many attributes and methods. It is very convenient to use. Of course, direct use will also encounter various compatibility issues. When learning the Video tag, the W3C official website gave such a warm reminder:

Playing videos in HTML is not easy!

You need to know a lot of skills to ensure that your video files can be played in all browsers(Internet Explorer, Chrome, Firefox, Safari, Opera) and all hardware(PC, Mac, iPad, iPhone).

This tip only came into contact with a series of video projects before I realized what this "not easy" means. On the mobile side, we need to know a lot of "a lot of skills" but it is not enough...

background

NutUI is a set of JD-style mobile component libraries that develop and serve enterprise-grade front-end and back-end products for mobile Web interfaces. There are now 50+ high-quality components, 1.9k stars have been obtained on GitHub, and NPM downloads have exceeded 14k. The company has been empowered to support 50+ projects, and 20+ external access projects. Interested students, come and scan the code to experience it!

image

To be honest, it has been some time since the release of the NutUI v2.2.2 version of the Video video component. We also received some user feedback on the NutUI communication group and GitHub. Here I want to talk to you about the development, use and encounter of the NutUI Video component. Problems and solutions.

First of all, the idea of ​​developing Vue video components originated from a mobile project. The project requirements are relatively simple, using the Vue technology stack, and only one video needs to be clicked to play, so when initially choosing to implement video playback, no third-party plug-ins were introduced. When investigating Vue's Video components at the beginning of the project development, it was found that the NutUI component library does not yet have video components. How can this be tolerated? So the NutUI Video component was born!

image

Preparation

Before developing, let's get to know the video tag again. I believe that when first acquainted with the <

<video controls width="250">
    <source src="videoname.webm" type="video/webm">
    <source src="videoname.mp4" type="video/mp4">
   Your browser does not support the video tag.
</video>

When the video can be played smoothly on the page, we only pay attention to the use of its attributes and parameters:

<video src="videofile.ogg" autoplay muted poster="posterimage.jpg">
  Sorry, your browser does not support embedded video
</video>

For example, in the above code, the video playback address src, the autoplay attribute autoplay, the mute attribute muted and the poster setting attribute poster are set.

In addition to basic optional attributes, the Video tag also supports global attributes and event attributes in HTML.

When we create a video in HTML, we can get the object attributes and methods of the Video tag, such as

  • currentTime The current playback position of the video(ie the current playback time, in seconds)
  • duration video length(playback length of the entire video, in seconds)
  • ended whether the video has finished playing
  • volume object attributes such as video playback volume
  • ......

There are also some object methods:

  • canPlayType() checks whether the browser can play the specified video type
  • load() to reload video elements
  • play() starts playing video
  • pause() pauses the currently playing video and other object methods.

Interested students can refer to the W3C and other related documents, and I will not repeat them here.

Function realization

Through the review of the Video tag, it can be said that the realization of video playback(only playback) in Vue is very simple, but if you want to "pass through" all the "hidden levels" on the mobile terminal, it can be said to be an impossible task. . Because even Video.js and Vue-video-player, which are widely circulated, still have many problems to be solved, we can only analyze the specific problems. So in the realization of NutUI Video component, we divided into two stages:

image

The first stage is the basic realization, complete the basic functions of video playback. The second stage is the realization of the advanced version of the custom control bar, which completes the custom development of operation items such as play, pause, and control bar.

Basic Implementation

1, the realization of attributes

For the implementation of attributes, I initially wanted to use a one-to-one correspondence to bind the attributes and throw them to the user. What the user operates is the native attributes of the video tag. However, considering the later iteration of the custom control bar, this method may not be conducive to management, so we still manage the operation attributes of the Video with the options object, and the video source is managed with the source attribute, which is managed in the form of a collection Video source information, which can support the configuration of multiple formats of video sources, in order to solve the compatibility problem of different device video formats:

<video ref="video" class="nut-videoplayer"
        :muted="options.muted"
        :autoplay="options.autoplay"
        :loop="options.loop"
        :poster="options.poster"
        :controls="options.controls"
        :preload="options.preload"
        >
        <source v-for="source in sources" :src="source.src" :type="source.type" :key="source.src" />
</video>

At this step, the user calls the component and configures the parameters to play the video normally:

<nut-video :sources="sources" :options="options"></nut-video>
 data() {
    return {
        sources:[{ src:'video.mp4', type:'video/mp4'}],
        options:{
            controls:true,
            autoplay:true,
            volume:0.6,
            poster:''
        },
    }
}

Effect demo

image

2, custom attributes

In addition to the basic properties of Video, in the basic version of the component, we also throw out some personalized property settings for the user such as:disabled prohibited operation, playsinline inline display, etc.

options:{autoplay:true, muted:true, disabled:true, playsinline:true, loop:true, controls:false}

The above configuration item specifies an example of a background image video that is automatically played in the line. It should be noted that the disable operation is currently only valid for automatic playback. In automatic playback, the user cannot operate the player, and clicking the player is invalid. While the playsinline attribute is displayed in the line, currently only the IOS side is compatible with individual Android devices. To fully achieve inline playback, specific problems or specific analysis is required.

Effect demonstration:

image

3, the realization of the event

In terms of event realization, the most important operations of the video are nothing more than three events:play, pause, and end of play, as well as an error event, which displays an error message when an error is reported.

image

When we use the native control bar of video, if we want to achieve playback, pause, and end of playback, we mainly rely on listening to video playback events.

//Monitor playback
this.videoElm.addEventListener('play',() => {
        this.state.playing = true;
        this.$emit('play', this.videoElm);
});
//Monitor pause
this.videoElm.addEventListener('pause',() => {
        this.state.playing = false;
        this.$emit('pause', this.videoElm);
});
//Monitor playback ends
this.videoElm.addEventListener('ended',this.playEnded);

The user calls the method as follows

<nut-video :sources="sources" :options="options" @play="play" @pause="pause" @playend="playend">
</nut-video>
methods:{
    play(elm) {console.log('play', elm);},
    pause(e) {console.log('pause');},
    playend(e) {alert('Play end');},
}

Effect demonstration:

image

As you can see from the video, when I click play, pause, and the end of the playback, the callback event will be triggered, and when the video ends, it will prompt the end of the playback.

Advanced version implementation--custom control implementation

If the basic version relies on the control bar of the native Video, then the realization of the custom control is the advanced version of mastering the autonomy of playback. Because the Video tag will have different default settings on different devices, it is difficult for us to control them, so customizing a set of our own video playback controls can to a certain extent avoid the problem of native controls being modified by default. Below, we take a look at its implementation.

1. Reconfiguration of the control bar

About restructuring the control bar, we can first look at the picture and analyze the elements needed for the custom control bar.

image

The icon above notes the elements needed for the control bar:

  • Play button
  • Current playing time
  • Overall time
  • Play control bar
  • Buffer time bar
  • Dragable play button
  • Mute control button
  • Full screen control buttons

Just refactor according to the elements of the above control bar, here is not much to do, just add the code.

<div class="nut-video-controller">
      <div class="control-play-btn" @click="play"></div> <!-- Play pause -->
      <div class="current-time">01:30</div> <!-- Current playing time -->
      <div class="progress-container"> <!-- Play Control Bar -->
        <div class="progress" ref="progressBar"> <!-- Overall playback time bar -->
          <div class="buffered" ></div> <!-- Buffer time bar -->
          <div class="video-ball" <!-- Drag the play button -->
            @touchmove.stop.prevent="touchSlidMove($event)"
            @touchstart.stop="touchSlidSrart($event)"
            @touchend.stop="touchSlidEnd($event)">
            <div class="move-handle"></div>
          </div>
          <div class="played" ref="playedBar"></div>
        </div>
      </div>
      <div class="duration-time">03:30</div> <!-- Overall time -->
      <div class="volume" @click="handleMuted"></div> <!-- Mute button -->
      <div class="fullscreen-icon" @click="fullScreen"></div> <!-- Full screen button -->
</div>

2. Initial configuration

After the control bar element reconstruction is complete, we need to first obtain the initial state of the Video element, custom control bar element, and user-configured attributes.

  • Get Video tags

    this.videoElm = this.$el.getElementsByTagName('video')[0];

Here we got the video tag, this step is very important, because all the subsequent operations are based on it.

  • Get custom control bar position

    const $player = this.$el;
    const $progress = this.$el.getElementsByClassName('progress')[0];
    //Player position
    this.player.$player = $player;
    this.progressBar.progressElm = $progress;
    this.progressBar.pos = $progress.getBoundingClientRect();
    this.videoSet.progress.width = Math.round($progress.getBoundingClientRect().width);

In the code, we get the control bar progressBar that we just reconstructed and define its position and width.

  • Initial property configuration

Initialization is to bind the attribute parameters set by the user to video, for example, to trigger a playback event during automatic playback settings, and to bind compatible attributes to video during inline playback settings, etc.

//Autoplay
if(this.options.autoplay) {
    this.videoElm.play();
}
//Play in line
if(this.options.playsinline) {
     this.videoElm.setAttribute('playsinline', this.options.playsinline);
     this.videoElm.setAttribute('webkit-playsinline', this.options.playsinline);
     this.videoElm.setAttribute('x5-playsinline', this.options.playsinline);
     this.videoElm.setAttribute('x5-video-player-type','h5');
     this.videoElm.setAttribute('x5-video-player-fullscreen', false);
}

3. Play and pause

For video playback and pause, we use the play() event control in the custom control bar, and use state.playing in data on the interface rendering.

play() {
    this.state.playing = !this.state.playing;
    if(this.videoElm) {
        //Play status
        if(this.state.playing) {
            try {
                this.videoElm.play();
                //monitor cache progress
                this.videoElm.addEventListener('progress', e => {this.getLoadTime();});
                //monitor playback progress
                this.videoElm.addEventListener('timeupdate', throttle(this.getPlayTime, 100, 1));
                //End of monitoring
                this.videoElm.addEventListener('ended', this.playEnded);
                this.$emit('play', this.videoElm);
            } catch(e) {
                this.handleError()
            }
        }
        //stop state
        else {
            this.videoElm.pause();
            this.$emit('pause', this.videoElm);
        }
    }
},

Trigger video.play() when the video is in the playback state, we will monitor the buffer progress, playback progress, and playback end status. When the video is paused, the video.pause() pause event will be triggered.

4, volume control

The volume control of a video is to set its volume after getting the Video element on the page, as follows.

volumeHandle() {
    this.videoElm.volume = this.state.vol;
}

5. Acquisition of playback time

The playing time is obtained according to duration and currentTime of video.

//Get play time
    getPlayTime() {
      const percent = this.videoElm.currentTime/this.videoElm.duration;
      this.videoSet.progress.current = Math.round(this.videoSet.progress.width * percent);

      //Assignment time
      this.videoSet.totalTime = this.timeFormat(this.videoElm.duration);
      this.videoSet.displayTime = this.timeFormat(this.videoElm.currentTime);
    },

6. Progress bar drag control

Speaking of the progress bar, through the layout of the control bar analyzed above, we know that it has a draggable button, and here we handle its touchmove and touchend events.

//Drag the playback progress
touchSlidMove(e) {
    let currentX = e.targetTouches[0].pageX;
    let offsetX = currentX-this.progressBar.pos.left;
    //boundary detection
    if(offsetX <= 0) {
        offsetX = 0;
    }
    if(offsetX >= this.videoSet.progress.width) {
        offsetX = this.videoSet.progress.width;
    }
    this.videoSet.progress.current = offsetX;
    let percent = this.videoSet.progress.current/this.videoSet.progress.width;
    this.videoElm.duration && this.setPlayTime(percent, this.videoElm.duration);

},
touchSlidEnd(e) {
    let currentX = e.changedTouches[0].pageX;
    let offsetX = currentX-this.progressBar.pos.left;
    this.videoSet.progress.current = offsetX;
    let percent = offsetX/this.videoSet.progress.width;
    this.videoElm.duration && this.setPlayTime(percent, this.videoElm.duration);
},
//Set manual play time
setPlayTime(percent, totalTime) {
    this.videoElm.currentTime = Math.floor(percent * totalTime);
},

Get the left position of the control bar at the beginning of the drag, and monitor the offset in real time, assign the value of the offset to the length of the this.videoSet.progress.width playback control bar, and convert it into time with a percentage. Set the current video playing time.

7, full screen control

Full screen and exit full screen We use state.fullScreen in data to control its button state, the default is false to indicate not full screen, when the user clicks the full screen button, set it to true and call enter full screen event element.webkitRequestFullScreen(), call document.webkitCancelFullScreen() when you click again to exit the full screen, and set state.fullScreen to false to change the style of the button icon.

fullScreen() {
    if(!this.state.fullScreen) {
        this.state.fullScreen = true;
        this.videoElm.webkitRequestFullScreen();
    } else {
        this.state.fullScreen = false;
        document.webkitCancelFullScreen();
    }
}

Custom control bar demo effect:

image

The above is the realization of the custom control bar, of course, there are other functions to be developed, and will continue to improve based on business and user feedback.

Problems & Solutions

After the components are developed, they can finally run in the project, but the problems that follow have appeared one after another. Here we summarize the problems and solutions we encountered in the project.

Autoplay problem

I believe that many people must have encountered the problem of automatic playback of video on the mobile terminal. After adding autoplay to the Video tag, the PC browser test is very good, and the test on the mobile phone fails. This is because of autoplay compatibility issues. The reasons for these problems may be:

  • Browser does not support this video format, it is recommended to use the three video formats MP4, WebM, Ogg
  • Out of user experience, to save traffic, the mobile terminal prohibits automatic playback
  • Video file is too large, loading time is too long or wrong

If you have to do the automatic playback function, you can refer to the following scheme:

  1. Check if the video format is correct, try to convert it to MP4, and the compression size is below 2M

  2. Autoplay is invalid in IOS device, you can add muted attribute:

  3. Simulate playback after user has touch screen operation.

    let video = document.getElementById("video");

     video.play();
  • It should be noted here that the analog playback must be performed after the user has operated, otherwise an error will be reported.
  • Simulated playback is invalid after the Android machine is loaded. The user must have touch screen operations to take effect, such as clicking, touching, sliding the screen, etc.
document.removeEventListener('touchstart', this.playVideo);
  1. If the automatic playback fails in WeChat, you can consider installing WeChat's JSSDK, and monitor the "WeixinJSBridgeReady" to control the automatic playback. The specific operations are as follows:

    document.addEventListener("WeixinJSBridgeReady", function() {

     document.getElementById('video').play();

    }, false);

  2. If the Android machine still cannot play automatically, you can consider the downgrade process, and the display control bar guides the user to click the play button to play.

Full screen playback problem

When playing video in full screen, we may encounter the situation that the screen is not occupied, and there will be black and white edges at the top and bottom. At this time, we can add style= "object-fit:fill;width:100%;height:100%;" To control the video to fill the screen.

Play in line

Play in the video line, that is, the video is partially embedded in the page, and it is played at the position where it is like the document stream. However, under the mobile terminal, Video video playback is the default full screen, so how to disable full screen?

  1. The IOS device can set the playsinline attribute on the Video tag. The compatible writing is as follows:

    <video muted src="video.mp4" autoplay

     webkit-playsinline
     playsinline
     x5-playsinline>

The above writing method can basically solve the problem of in-line playback in IOS. x5-playsinline can make some Android machines compatible, but after adding this attribute, there can no longer be x5-video-player-type='h5' and x5 -video-player-fullscreen='true', otherwise it will default to full screen.

  1. Canvas analog video playback

If the above solutions are not sufficient for Android devices, you can try using canvas to simulate video playback, hide the video tag on the page, and draw the video on the canvas by monitoring the playback, pause, and playback end events.

initCanvas() {
    //Get video
    let TestVideo = document.getElementById('videoPlay');
    let videoW = TestVideo.offsetWidth;
    let videoH = TestVideo.offsetHeight;
    //Get canvas
     let TestCanvas = document.getElementById('videoCanvas');
     //Set the canvas
     let TestCanvas2D = TestCanvas.getContext('2d');
     //Set the setinterval timer
     let TestVideoTimer = null;
     //Monitor playback
     TestVideo.addEventListener('play', function() {
         TestVideoTimer = setInterval(function() {
             TestCanvas2D.drawImage(TestVideo,0,0,320,180);
         }, 20);
     }, false);
    //Monitor pause
     TestVideo.addEventListener('pause',function() {
         clearInterval(TestVideoTimer);
     }, false);
     //End of monitoring
     TestVideo.addEventListener('ended', function() {
            clearInterval(TestVideoTimer);
    }, false);

 }

Although using canvas to simulate video can achieve in-line display, the effect is not very satisfactory. The clarity of the video playback is not high, and there will be problems with stuttering. It may also be that the video source I used when experimenting with this method was compressed. The picture quality is very poor, there is a little problem with the mobile terminal to control the size of the canvas. Interested students can try using canvas to simulate the video playback.

issue

After the release, we have also received feedback on some issues one after another, and we have also checked and repaired these issues one by one.

The issue is as follows:

image

1, the video component running console will report an error

This problem is caused by the code left uncommented when developing the custom control. The new version has been fixed.

2. Video source asynchronous switching

After the basic version was released, there was feedback in the NutUI communication group that when the video source was switched asynchronously, the video could not be played. That is because the video address has not been monitored when switching, and it can be solved by adding the monitoring event to the component and loading it again.

The method is optimized as follows:

 watch:{
    sources:{
        handler(newValue, oldValue) {
            if(newValue && oldValue && newValue != oldValue) {
                this.$nextTick(() => {
                    this.videoElm.load()
                })
            }
        },
        immediate:true
    },

},

This method has been launched with the new version, you can experience it after updating the version.

Thank you for your feedback, and I hope you can provide more valuable comments to help us catch bugs together so that this video component can go further.

to sum up

Although the first version of the Video component has been released, the function is only based on the packaging of the native Video tag. In the face of complex compatibility issues on the mobile terminal, it needs to be constantly polished. The development of the custom control bar is currently in the experimental stage, and I hope that in the near future there will be a set of compatible native and custom Video components to meet you. If you have any good suggestions for the development of NutUI Video, please leave a message to participate in the development and design of NutUI Video! The road to the development of video components on the mobile terminal is difficult and long. Let us step by step~