Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HLS Interstitials support #6591

Open
wants to merge 11 commits into
base: master
Choose a base branch
from
Open

HLS Interstitials support #6591

wants to merge 11 commits into from

Conversation

robwalch
Copy link
Collaborator

@robwalch robwalch commented Jul 30, 2024

This PR will...

Add support for HLS Interstitials parsing, playback, and events.

Design Details

The InterstitialsController handles HLS and media input and events for the primary HLS asset. It passes date range data to the scheduler to produce an array of Interstitial events and an array of event and primary items called the schedule. InterstitialsController has:

  • an InterstitialsSchedule which produces Interstitial events and a schedule of items on level update. The InterstitialsSchedule has
    • an array of InterstitialEvent objects. Each event contains properties and getters for finding its place on the playback timeline as well as an array of interstitial assets
  • an array of HLSAssetPlayer instances call the player queue. Each HLSAssetPlayer wraps a child instance of Hls used to preload and stream Interstitial assets.
  • the responsibility of loading asset lists. The InterstitialsController adds a loader to the corresponding InterstitialEvent while loading. The loader is removed and replaced by the asset list response once the request is complete.
  • the responsibility of scheduling Interstitial playback and maintaining the current playing and buffering schedule items and assets. The schedule advances according to primary currentTime changes on timeupdate and seeking events, advancement of the combined buffer on media append, and asset playback and buffer advancement and completion.

API Enhancements

Configuration options

  • Interstitials parsing and playback are enabled via config option interstitialsController. Setting this to null turns off Interstitial parsing and playback.
  • To turn off Interstitial playback without removing parsing or schedule updates and buffered-to events, set config option enableInterstitialPlayback to false (allowing for custom playout and ad managers)
  • Config option interstitialAssetListLoadPolicy defines the loading policy of X-ASSET-LIST JSON
  • Several config options have been added specifically for Interstitial asset player instances:
    • primarySessionId identifies the parent player session that spawned the asset player (read from hls.sessionId)
    • assetPlayerId is used to identify logs from asset players
    • timelineOffset is used to offset MSE appends of Interstitial content (not all Interstitial assets are appended inline at offsets; most require a MediaSource reset)

New top-level API on Hls instances

  • bufferedToEnd getter returns a boolean indicating if EOS has been appended (media is buffered from currentTime to end of stream)
  • bufferingEnabled getter returns a boolean indicating whether fragment loading has been toggled with pauseBuffering() and resumeBuffering()
  • interstitialsManager getter returns an InterstitialsManager (or null). The InterstitialsManager is an interface that provides access to Interstitial program and timeline data as well as methods for seeking across items and skipping Interstitial events.
  • latestLevelDetails getter returns the LevelDetails of the most up-to-date HLS variant Playlist data
  • sessionId getter returns the session UUID assigned to the Hls instance
    • The sessionId value is used when assigning a _HLS_primary_id query parameter to interstitial requests
  • startLoad() now includes a second optional argument to skip seeking on start (otherwise, HLS.js seeks following to the first optional startPosition argument on append)
  • hasEnoughToStart getter returns whether enough is buffered to seek to start position (fix: media buffer is not empty but video element buffer is empty #6571)
  • startPosition getter returns the resolved startPosition that playback will begin at once media is appended
  • transferMedia() method detaches and returns MediaSource and SourceBuffers non-destructively
  • url getter returns the URL resolved from loadSource(url)

Updated top-level API on Hls instances

New Events for Interstitials

  • ASSET_LIST_LOADING when a request is made for an X-ASSET-LIST JSON object
  • ASSET_LIST_LOADED when a response is received for an X-ASSET-LIST JSON object
  • INTERSTITIALS_UPDATED when Interstitials are added, removed, or the schedule is updated following a variant playlist update or updated asset durations from X-ASSET-LIST JSON or asset playlist and media parsing
  • INTERSTITIALS_BUFFERED_TO_BOUNDARY when the forward buffer reaches the boundary of the following schedule item (Interstitial event or primary segment)
  • INTERSTITIAL_ASSET_PLAYER_CREATED when an asset player instance is created to stream an Interstitial asset (will always be before attaching media to the asset player)
  • INTERSTITIAL_STARTED when streaming of an Interstitial event containing one or more assets has begun (may occur before X-ASSET-LIST JSON is loaded or playback has started)
  • INTERSTITIAL_ENDED when streaming of an Interstitial event containing one or more assets has ended - before resuming primary or starting the next event
  • INTERSTITIAL_ASSET_STARTED when streaming of an Interstitial asset has begun (following the beginning of the event or the end of the last asset)
  • INTERSTITIAL_ASSET_ENDED when streaming of an Interstitial asset has ended (before the next asset or the event ending)
  • INTERSTITIAL_ASSET_ERROR when an error occurs starting or streaming an Interstitial asset (this can include non-fatal errors such as stalling and errors that will end streaming of the asset, resulting in the schedule advancing to the next asset or fallback to primary)
  • INTERSTITIALS_PRIMARY_RESUMED when playback of primary content has begun or resumed from an Interstitial event
  • BUFFERED_TO_END when the last audio and video segments in the playlist have been appended (EOS signaled on all SourceBuffers)
  • AUDIO_TRACK_UPDATED similar to LEVEL_UPDATED fired for any update to audio group playlists
  • SUBTITLE_TRACK_UPDATED similar to LEVEL_UPDATED fired for any update to subtitle group playlists

Updated Events

  • MEDIA_ATTACHING, MEDIA_ATTACHED, MEDIA_DETACHING, and MEDIA_DETACHED include additional information (depending on whether media is being transferred)

New Errors for Interstitials

  • Type: NETWORK_ERROR
    • details: ASSET_LIST_LOAD_ERROR network error loading asset list
    • details: ASSET_LIST_LOAD_TIMEOUT network timeout error loading asset list
    • details: ASSET_LIST_PARSING_ERROR asset list was not valid JSON or missing required data
  • type: OTHER_ERROR details: INTERSTITIAL_ASSET_ITEM_ERROR an issue interrupted or prevented asset playback. This will result in skipping the remainder of the asset or falling back to primary content. The event error will contain more details. This type of error differs from the INTERSTITIAL_ASSET_ERROR events forwarded from asset player errors.

Build Constants

  • Interstitials is removed from the hls.light.js build by the __USE_INTERSTITALS__ build const/directive.

Known Issues

Should address before release (in order of priority):

  • InterstitialsManager primary, playout and integrated durations are non-finite for Live streams. These should be finite for easier mapping of controls. resolved with 5842cc2
  • hls.interstitialsManager.skip() does not skip Interstitial events that do not reset MediaSource (where InterstitialEvent appendInPlace is true) resolved with 0021cfd e84bc30 2c3c372
  • In some cases the MediaSource is reset at the end of a post roll or playback ends at the end of the post roll, but attempting to play from that state does not restart the program.

No plan to address before release:

  • There is no API provided for defining client-side Interstitials (file an issue stating your interest and use case).
  • Primary subtitles TextTrack cues are not edited to avoid overlapping with Interstitial events scheduled without MediaSource reset (where InterstitialEvent appendInPlace is true). Note that this only applies to Interstitials that overlap and replace primary segments completely which is not the case for most Interstitials. If you have content where this is a concern, please file an issue.

Resolves issues:

Checklist

  • changes have been done against master branch, and PR does not conflict
  • new unit / functional tests have been added (whenever applicable)
  • API or design changes are documented in API.md

robwalch and others added 5 commits July 30, 2024 17:08
…grated duration

(cherry picked from commit 161bf44d0186a263bc82299fc09e52b913f2d2f8)
…itialsManager.skip()`

(cherry picked from commit 928064edb0d802d1e6513bd509e65ef5c584f2e9)
(cherry picked from commit ffa6612c848033ea913c020e0e5355db710ff431)
(cherry picked from commit 1cc169ce64b26aff31929a510b8ff157558c9bb5)
(cherry picked from commit 34ab0ed5016a3c18e330ab136fd731c3d5c95fcc)
@robwalch robwalch mentioned this pull request Aug 7, 2024
3 tasks
robwalch and others added 5 commits August 7, 2024 18:29
…nterstitials

Make criteria for "append in place" more flexible (does not require snapping and allows for resume offset matching duration)

(cherry picked from commit 856dfb09aeadc99c5f9788ca5e55310fd8fa4d3d)
(cherry picked from commit 446ef3cee40ab0f7337fe12c905905349f24d8f7)
…ce" mode

Improve Interstitial logging and log media transfer and "appendInPlace" details
Fix media transfer for Interstitials in "appendInPlace" mode, with abutting timing, and snap-in

(cherry picked from commit 9911bc4b8f9d59a4843affd7b889b11aab2c0117)
Fix asset player startPosition with append-in-place timelineOffset
Fix issues with item and asset lookup using Array.indexOf when object refererenes are replaced by Live updates
Fix cue removal by asset players

(cherry picked from commit eb44d8699e4fa48becdba377aa4a82ece6d1f6f3)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
1 participant