Implementing Cue Points Programmatically

Video Cloud cue points

In this section of the document, you will learn to set catalog cue points and listen for catalog cue point event dispatches.

Setting Video Cloud cue points

Video Cloud cue points can be associated with a video using the Video Cloud Studio, and a number of other ways, detailed in this document: Working with Cue Points in the Media Module.

Acting on Video Cloud cue points - video statically bound to the player

In this section of the document, you will learn to process catalog cue points when the video has been statically bound to the player, meaning the video has been loaded in the player either in Studio or using the Player Management API directly.

To prevent a race condition of trying to process cue points before they are loaded, you need to use the loadedmetadata event to be dispatched before dealing with the cue points. Once the proper text track has been read, use the oncuechange event to listen for cue points events to be dispatched.

The following code shows listening for cue points and displaying the data from the cue point. Note that in this example the video is statically bound to the Player.

• Line 11: Create a paragraph element as a location in which to inject dynamically created HTML.
• Lines 18,30: Use the one() method to add an event listener for the loadedmetadata event only once. The event handler function is defined here anonymously.
• Line 19: Retrieve the TextTracks array using the textTracks() method, then assign the zeroth element, which holds the cue points, to the variable tt. Note that it may be possible in some implementations that the cue points could be in a different array element. See the Find correct track section below for more information.
• Lines 20,28: Sets the event handler function for when the oncuechangeevent is dispatched.
• Line 21: Check to be sure that you are getting the first (zeroth array element) cue point dispatch. Without this condition, you would see that each cue point would be acted upon twice. Note that if you are using cue points that have durations that overlap this condition would have to be different.
• Lines 22-26: Dynamically create HTML using information from the cue point and inject into the HTML page.
• Line 29: Play the video.

<video-js id="myPlayerID"
  data-account="1507807800001"
  data-player="zN3V18ZPEu"
  data-embed="default"
  controls=""
  data-video-id="1507781667001"
  data-playlist-id=""
  data-application-id=""
  width="960" height="540"></video-js>

<p id="insertionPoint"></p>

<script src="https://players.brightcove.net/1507807800001/zN3V18ZPEu_default/index.min.js"></script>

<script>
  videojs.getPlayer('myPlayerID').ready(function() {
    var player = this;
    player.one("loadedmetadata", function () {
      var tt = player.textTracks()[0];
      tt.oncuechange = function () {
        if (tt.activeCues[0] !== undefined) {
          var dynamicHTML = "id: " + tt.activeCues[0].id + ", ";
          dynamicHTML += "text: " + tt.activeCues[0].text + ", ";
          dynamicHTML += "startTime: " + tt.activeCues[0].startTime + ",  ";
          dynamicHTML += "endTime: " + tt.activeCues[0].endTime;
          document.getElementById("insertionPoint").innerHTML += dynamicHTML + "<br/>";
        }
      }
      player.play();
    });
  });
</script>

Acting on Video Cloud cue points - dynamically load video in the player

In this section of the document, you will learn to process catalog cue points when the video is dynamically loaded into the player using the catalog.getVideo() and catalog.load() methods.

When you use the player catalog to get and load the video, processing cue points is bit easier than when you use a statically bound video, as you do not need to use the loadedmetadata event.

• Line 11: Create a paragraph element as a location in which to inject dynamically created HTML.
• Lines 17,31: Use the catalog.getVideo() method to retrieve a video. The callback function is defined here anonymously.
• Line 19: Use the catalog.load() method to load the video into the player.
• Line 21: Retrieve the TextTracks array using the textTracks() method, then assign the zeroth element, which holds the cue points, to the variable tt. Note that it may be possible in some implementations that the cue points could be in a different array element. See the Find correct track section below for more information.
• Lines 22-30: Sets the event handler function for when the oncuechangeevent is dispatched.
• Line 23: Check to be sure the first (zeroth array element) cue point is defined. Without this condition you would see that each cue point would be acted upon twice, the second time with no defined elements in the activecues array (since the second cue change is for cue point end). Note that if you are using cue points that have durations that overlap this condition would have to be different.
• Lines 24-28: Dynamically create HTML using information from the cue point and inject into the HTML page.
• Line 32: Play the video.

<video-js id="myPlayerID"
  data-account="1507807800001"
  data-player="zN3V18ZPEu"
  data-embed="default"
  controls=""
  data-video-id=""
  data-playlist-id=""
  data-application-id=""
  width="960" height="540"></video-js>

<p id="insertionPoint"></p>

<script src="https://players.brightcove.net/1507807800001/zN3V18ZPEu_default/index.min.js"></script>
  <script>
  videojs.getPlayer('myPlayerID').ready(function() {
    var player = this;
    player.catalog.getVideo('1507781667001', function (error, video) {
      //deal with error
      player.catalog.load(video);
      player.one("loadedmetadata", function () {
        var tt = player.textTracks()[0];
        tt.oncuechange = function () {
          if (tt.activeCues[0] !== undefined) {
            var dynamicHTML = "id: " + tt.activeCues[0].id + ", ";
            dynamicHTML += "text: " + tt.activeCues[0].text + ", ";
            dynamicHTML += "startTime: " + tt.activeCues[0].startTime + ",  ";
            dynamicHTML += "endTime: " + tt.activeCues[0].endTime;
            document.getElementById("insertionPoint").innerHTML += dynamicHTML + "<br/>";
          }
        }
      });
      player.play();
    });
  });
</script>

Retrieve all Video Cloud cue point info

As you may have noticed not all Video Cloud cue point information is directly available to you from the activecues array. This is easily remedied by retrieving the desired info from the mediainfo property.

The basic approach of this solution is:

1. Wait for the loadstart event for the mediainfo property to be populated.
2. Assign the cue_points array from the mediainfo property to a variable. This variable contains complete Video Cloud cue point information.
3. On a cue point event, retrieve the corresponding cue point data based on the time property value. This will be done using a helper function that grabs an object out of an array based on a property value in the object.
4. Use the data from the cue point.

The following image shows the entire cue point array (top-left), the single cue point data collection (top-right), and one property from that single cue point data collection (bottom-right).

In the snippet below only the new/changed code from examples above will be explained.

• Lines 452-463: Include the function that will be used to extract a single cue point data collection from the array of all cue points. Note that you pass it the array of all cue points, the property in which you are searching for a specific value, and lastly the value you are searching for.
• Line 432: Listens for the loadstart event. When your video starts to load, then the mediainfo property is populated.
• Line 434: Assign the array of all Video Cloud cue points to a variable.
• Line 443: In the cue point dispatch event handler, assign the specific cue point's data collection to a variable. This is where the function mentioned in the first bullet point is called. The arguments used are:
  • cuePointAra: The entire collection of Video Cloud cue points.
  • 'time': The property in which to search for a value.
  • tt.activeCues[0].startTime: The start time of the cue point currently being handled in the cue point dispatch event handler.
• Lines 444-445: Debugging console.log() method calls, which should be removed in production code.

<script>
  videojs.getPlayer('myPlayerID').ready(function() {
    var myPlayer = this,
      cuePointAra = [],
      allCuePointData;
    myPlayer.on('loadstart', function () {
      //console.log('mediainfo', myPlayer.mediainfo);
      cuePointAra = myPlayer.mediainfo.cue_points;
      var tt = myPlayer.textTracks()[0];
      tt.oncuechange = function () {
        if (tt.activeCues[0] !== undefined) {
          var dynamicHTML = "id: " + tt.activeCues[0].id + ", ";
          dynamicHTML += "text: " + tt.activeCues[0].text + ", ";
          dynamicHTML += "startTime: " + tt.activeCues[0].startTime + ",  ";
          dynamicHTML += "endTime: " + tt.activeCues[0].endTime;
          document.getElementById("insertionPoint").innerHTML += dynamicHTML + "<br/>";
          allCuePointData = getSubArray(cuePointAra, 'time', tt.activeCues[0].startTime);
          console.log('cue point data:', allCuePointData);
          console.log('cue point metadata:', allCuePointData[0].metadata);
        }
      }
      myPlayer.play();
      myPlayer.muted(true);
    });

    function getSubArray(targetArray, objProperty, value) {
      var i, totalItems = targetArray.length,
        objFound = false,
        idxArr = [];
      for (i = 0; i < totalItems; i++) {
        if (targetArray[i][objProperty] === value) {
          objFound = true;
          idxArr.push(targetArray[i]);
        }
      }
      return idxArr;
    };
  });
</script>

Programmatic cue points

It is possible to programmatically add cue points. The key method comes from the HTML5 VTTCue interface. You can create cue points use the following syntax:

new VTTCue( startTime, endTime, text )

The logic of processing the cue points is very similar to previous examples, so only the code different from that used above will be detailed.

• Lines 43,44: Adds a textTrack element to the player using the player's addRemoteTextTrack() method. Specify the kind to be metadata, and a label, in this case Timed Cue Point.

• Lines 45-48: Wait 10 milliseconds before creating two cue points using the player's addCue() method. The cue points are instantiated using HTML5's VTTCue() constructor.

• Line 22: Listens for the addtrack event.
• Line 23: Retrieves all text tracks associated with the player.
• Line 24: Sets the attLength variable to the number of text tracks. This value will be used in a loop in the next code segment.
• Lines 25-30: Loop over each text track checking for the label to be equal to the correct value. When found, assign the current text track to a variable and break out of the loop.

<video-js id="myPlayerID"
  data-video-id="4607746980001"
  data-account="1507807800001"
  data-player="default"
  data-embed="default"
  width="640" height="360"
  controls=""></video-js>

<script src="https://players.brightcove.net/1507807800001/default_default/index.min.js"></script>

<p id="insertionPoint"></p>

<script type="text/javascript">
  videojs.getPlayer("myPlayerID").ready(function () {
    var myPlayer = this,
      textTrack = [],
      allTextTacks,
      attLength,
      tt;
    myPlayer.one("loadedmetadata", function () {
      myPlayer.textTracks().addEventListener('addtrack', function () {
        allTextTacks = myPlayer.textTracks();
        attLength = allTextTacks.length;
        for (var i = 0; i < attLength; i++) {
          if (allTextTacks[i].label === 'Timed Cue Point') {
            tt = allTextTacks[i];
            break;
          }
        }
        tt.oncuechange = function () {
          if (tt.activeCues[0] !== undefined) {
            var dynamicHTML = "id: " + tt.activeCues[0].id + ", ";
            dynamicHTML += "text: <strong>" + tt.activeCues[0].text + "</strong>, ";
            dynamicHTML += "startTime: <strong>" + tt.activeCues[0].startTime + "</strong>,  ";
            dynamicHTML += "endTime: <strong>" + tt.activeCues[0].endTime + "</strong>";
            document.getElementById("insertionPoint").innerHTML += dynamicHTML + "<br/><br/>";
          } else {
            document.getElementById("insertionPoint").innerHTML += "Cue point duration over" + "<br/><br/>";
          }
        }; //end oncuechange
      }); // end playing
      textTrack = myPlayer.addRemoteTextTrack({kind: 'metadata', label: 'Timed Cue Point', mode: 'hidden'}, false);
      textTrack.track.mode = 'hidden';
      setTimeout(function(){
        textTrack.track.addCue(new window.VTTCue(2, 5, 'cue point 1 text'));
        textTrack.track.addCue(new window.VTTCue(10, 15, 'cue point 2 text'));
      }, 10);
    }); //end on loadedmetadata
  }); //end ready
</script>

The code generates output as shown in the following screenshot. Note the id values are not supplied a value when using the VTTCue() constructor method.