Synchronising multiple media elements - HTML 5

Each media element can have a MediaController. A MediaController is an object that coordinates the playback of multiple media elements, for instance so that a sign-language interpreter track can be overlaid on a video track, with the two being kept in sync.

Media elements with a MediaController are said to be slaved to their controller. The MediaController modifies the playback rate and the playback volume of each of the media elements slaved to it, and ensures that when any of its slaved media elements unexpectedly stall, the others are stopped at the same time.

4.8.10.11.2 Media controllers

A media element can have a current media controller, which is a MediaController object. When a media element is created without a mediagroup attribute, it does not have a current media controller. (If it is created with such an attribute, then that attribute initializes the current media controller, as defined below.)

The slaved media elements of a MediaController are the media elements whose current media controller is that MediaController. All the slaved media elements of a MediaController must use the same clock for their definition of their media timeline's unit time. When the user agent is required to act on each slaved media element in turn, they must be processed in the order that they were last associated with the MediaController.

The controller attribute on a media element, on getting, must return the element's current media controller, if any, or null otherwise. On setting, the user agent must run the following steps:

Let m be the media element in question.
Let old controller be m's current media controller, if it currently has one, and null otherwise.
Let new controller be null.
Let m have no current media controller, if it currently has one.
Remove the element's mediagroup content attribute, if any.
If the new value is null, then jump to the update controllers step below.
Let m's current media controller be the new value.
Let new controller be m's current media controller.
Bring the media element up to speed with its new media controller.
Update controllers: If old controller and new controller are the same (whether both null or both the same controller) then abort these steps.
If old controller is not null and still has one or more slaved media elements, then report the controller state for old controller.
If new controller is not null, then report the controller state for new controller.

The MediaController() constructor, when invoked, must return a newly created MediaController object.

The readyState attribute must return the value to which it was most recently set. When the MediaController object is created, the attribute must be set to the value 0 (HAVE_NOTHING). The value is updated by the report the controller state algorithm below.

The seekable attribute must return a new static normalized TimeRanges object that represents the intersection of the ranges of the media resources of the slaved media elements that the user agent is able to seek to, at the time the attribute is evaluated.

The buffered attribute must return a new static normalized TimeRanges object that represents the intersection of the ranges of the media resources of the slaved media elements that the user agent has buffered, at the time the attribute is evaluated. Users agents must accurately determine the ranges available, even for media streams where this can only be determined by tedious inspection.

The duration attribute must return the media controller duration.

Every 15 to 250ms, or whenever the MediaController's media controller duration changes, whichever happens least often, the user agent must queue a task to fire a simple event named durationchange at the MediaController. If the MediaController's media controller duration decreases such that the media controller position is greater than the media controller duration, the user agent must immediately seek the media controller to media controller duration.

The currentTime attribute must return the media controller position on getting, and on setting must seek the media controller to the new value.

Every 15 to 250ms, or whenever the MediaController's media controller position changes, whichever happens least often, the user agent must queue a task to fire a simple event named timeupdate at the MediaController.

When a MediaController is created it is a playing media controller. It can be changed into a paused media controller and back either via the user agent's user interface (when the element is exposing a user interface to the user) or by script using the APIs defined in this section (see below).

The paused attribute must return true if the MediaController object is a paused media controller, and false otherwise.

When the pause() method is invoked, if the MediaController is a playing media controller then the user agent must change the MediaController into a paused media controller, queue a task to fire a simple event named pause at the MediaController, and then report the controller state of the MediaController.

When the unpause() method is invoked, if the MediaController is a paused media controller, the user agent must change the MediaController into a playing media controller, queue a task to fire a simple event named play at the MediaController, and then report the controller state of the MediaController.

When the play() method is invoked, the user agent must invoke the play method of each slaved media element in turn, and then invoke the unpause method of the MediaController.

The playbackState attribute must return the value to which it was most recently set. When the MediaController object is created, the attribute must be set to the value "waiting". The value is updated by the report the controller state algorithm below.

The played attribute must return a new static normalized TimeRanges object that represents the union of the ranges of points on the media timelines of the media resources of the slaved media elements that the user agent has so far reached through the usual monotonic increase of their current playback positions during normal playback, at the time the attribute is evaluated.

A MediaController has a media controller default playback rate and a media controller playback rate, which must both be set to 1.0 when the MediaController object is created.

The defaultPlaybackRate attribute, on getting, must return the MediaController's media controller default playback rate, and on setting, must set the MediaController's media controller default playback rate to the new value, then queue a task to fire a simple event named ratechange at the MediaController.

The playbackRate attribute, on getting, must return the MediaController's media controller playback rate, and on setting, must set the MediaController's media controller playback rate to the new value, then queue a task to fire a simple event named ratechange at the MediaController.

A MediaController has a media controller volume multiplier, which must be set to 1.0 when the MediaController object is created, and a media controller mute override, much must initially be false.

The volume attribute, on getting, must return the MediaController's media controller volume multiplier, and on setting, if the new value is in the range 0.0 to 1.0 inclusive, must set the MediaController's media controller volume multiplier to the new value and queue a task to fire a simple event named volumechange at the MediaController. If the new value is outside the range 0.0 to 1.0 inclusive, then, on setting, an IndexSizeError exception must be thrown instead.

The muted attribute, on getting, must return the MediaController's media controller mute override, and on setting, must set the MediaController's media controller mute override to the new value and queue a task to fire a simple event named volumechange at the MediaController.

The media resources of all the slaved media elements of a MediaController have a defined temporal relationship which provides relative offsets between the zero time of each such media resource: for media resources with a timeline offset, their relative offsets are the difference between their timeline offset; the zero times of all the media resources without a timeline offset are not offset from each other (i.e. the origins of their timelines are cotemporal); and finally, the zero time of the media resource with the earliest timeline offset (if any) is not offset from the zero times of the media resources without a timeline offset (i.e. the origins of media resources without a timeline offset are further cotemporal with the earliest defined point on the timeline of the media resource with the earliest timeline offset).

The media resource end position of a media resource in a media element is defined as follows: if the media resource has a finite and known duration, the media resource end position is the duration of the media resource's timeline (the last defined position on that timeline); otherwise, the media resource's duration is infinite or unknown, and the media resource end position is the time of the last frame of media data currently available for that media resource.

Each MediaController also has its own defined timeline. On this timeline, all the media resources of all the slaved media elements of the MediaController are temporally aligned according to their defined offsets. The media controller duration of that MediaController is the time from the earliest earliest possible position, relative to this MediaController timeline, of any of the media resources of the slaved media elements of the MediaController, to the time of the latest media resource end position of the media resources of the slaved media elements of the MediaController, again relative to this MediaController timeline.

Each MediaController has a media controller position. This is the time on the MediaController's timeline at which the user agent is trying to play the slaved media elements. When a MediaController is created, its media controller position is initially zero.

When the user agent is to bring a media element up to speed with its new media controller, it must seek that media element to the MediaController's media controller position relative to the media element's timeline.

When the user agent is to seek the media controller to a particular new playback position, it must follow these steps:

If the new playback position is less than zero, then set it to zero.
If the new playback position is greater than the media controller duration, then set it to the media controller duration.
Set the media controller position to the new playback position.
Seek each slaved media element to the new playback position relative to the media element timeline.

A MediaController is a blocked media controller if the MediaController is a paused media controller, or if any of its slaved media elements are blocked media elements, or if any of its slaved media elements whose autoplaying flag is true still have their paused attribute set to true, or if all of its slaved media elements have their paused attribute set to true.

A media element is blocked on its media controller if the MediaController is a blocked media controller, or if its media controller position is either before the media resource's earliest possible position relative to the MediaController's timeline or after the end of the media resource relative to the MediaController's timeline.

When a MediaController is not a blocked media controller and it has at least one slaved media element whose Document is a fully active Document, the MediaController's media controller position must increase monotonically at media controller playback rate units of time on the MediaController's timeline per unit time of the clock used by its slaved media elements.

When the zero point on the timeline of a MediaController moves relative to the timelines of the slaved media elements by a time difference ΔT, the MediaController's media controller position must be decremented by ΔT.

In some situations, e.g. when playing back a live stream without buffering anything, the media controller position would increase monotonically as described above at the same rate as the ΔT described in the previous paragraph decreases it, with the end result that for all intents and purposes, the media controller position would appear to remain constant (probably with the value 0).

A MediaController has a most recently reported readiness state, which is a number from 0 to 4 derived from the numbers used for the media element readyState attribute, and a most recently reported playback state, which is either playing, waiting, or ended.

When a MediaController is created, its most recently reported readiness state must be set to 0, and its most recently reported playback state must be set to waiting.

When a user agent is required to report the controller state for a MediaController, the user agent must run the following steps:

If the MediaController has no slaved media elements, let new readiness state be 0.

Otherwise, let it have the lowest value of the readyState IDL attributes of all of its slaved media elements.
If the MediaController's most recently reported readiness state is less than the new readiness state, then run these substeps:
1. Let next state be the MediaController's most recently reported readiness state.
2. Loop: Increment next state by one.
3. Queue a task to run the following steps:
  1. Set the MediaController's readyState attribute to the value next state.
  2. Fire a simple event at the MediaController object, whose name is the event name corresponding to the value of next state given in the table below.
4. If next state is less than new readiness state, then return to the step labeled loop.
Otherwise, if the MediaController's most recently reported readiness state is greater than new readiness state then queue a task to fire a simple event at the MediaController object, whose name is the event name corresponding to the value of new readiness state given in the table below.

Value of new readiness state Event name

0 emptied
1 loadedmetadata
2 loadeddata
3 canplay
4 canplaythrough
Let the MediaController's most recently reported readiness state be new readiness state.
Initialize new playback state by setting it to the state given for the first matching condition from the following list:

If the MediaController has no slaved media elements

Let new playback state be waiting.

If all of the MediaController's slaved media elements have ended playback and the media controller playback rate is positive or zero

Let new playback state be ended.

If the MediaController is a blocked media controller

Let new playback state be waiting.

Otherwise

Let new playback state be playing.
If the MediaController's most recently reported playback state is not equal to new playback state and the new playback state is ended, then queue a task that, if the MediaController object is a playing media controller, and all of the MediaController's slaved media elements have still ended playback, and the media controller playback rate is still positive or zero, changes the MediaController object to a paused media controller and then fires a simple event named pause at the MediaController object.
If the MediaController's most recently reported playback state is not equal to new playback state then queue a task to run the following steps:
1. Set the MediaController's playbackState attribute to the value given in the second column of the row of the following table whose first column contains the new playback state.
2. Fire a simple event at the MediaController object, whose name is the value given in the third column of the row of the following table whose first column contains the new playback state.
New playback state New value for playbackState Event name

playing "playing" playing
waiting "waiting" waiting
ended "ended" ended
Let the MediaController's most recently reported playback state be new playback state.

The following are the event handlers (and their corresponding event handler event types) that must be supported, as IDL attributes, by all objects implementing the MediaController interface:

Event handler	Event handler event type
`onemptied`	`emptied`
`onloadedmetadata`	`loadedmetadata`
`onloadeddata`	`loadeddata`
`oncanplay`	`canplay`
`oncanplaythrough`	`canplaythrough`
`onplaying`	`playing`
`onended`	`ended`
`onwaiting`	`waiting`
`ondurationchange`	`durationchange`
`ontimeupdate`	`timeupdate`
`onplay`	`play`
`onpause`	`pause`
`onratechange`	`ratechange`
`onvolumechange`	`volumechange`

The task source for the tasks listed in this section is the DOM manipulation task source.

4.8.10.11.3 Assigning a media controller declaratively

The mediagroup content attribute on media elements can be used to link multiple media elements together by implicitly creating a MediaController. The value is text; media elements with the same value are automatically linked by the user agent.

4.8.10.11. Synchronising multiple media elements

4.8.10.11.1 Introduction

4.8.10.11.2 Media controllers

4.8.10.11.3 Assigning a media controller declaratively