Text to Video with Scene-Level Background Music

This example demonstrates how to enable or disable background music at the scene level. This allows you to have different background music settings for different scenes within the same video.

Overview

This example covers:

Getting an access token
Setting background music at the video level
Enabling/disabling background music for specific scenes
Understanding scene-level music control
Creating videos with selective music
Monitoring job status and retrieving the final video

Node.js Example

Prerequisites

npm install axios

Complete Code

import axios from "axios";

const API_BASE_URL = "https://api.pictory.ai/pictoryapis";
const CLIENT_ID = "YOUR_CLIENT_ID";
const CLIENT_SECRET = "YOUR_CLIENT_SECRET";

const STORY_TEXT_1 = "AI is poised to significantly impact educators and course creators on social media.";
const STORY_TEXT_2 =
  "By automating tasks like content generation, visual design, and video editing, AI will save time and enhance consistency.";
const STORY_TEXT_3 = "This allows creators to focus on higher-level strategies and ensures a cohesive brand presence.";

async function createTextToVideoWithSceneBackgroundMusic() {
  try {
    // Step 1: Get Access Token
    console.log("Step 1: Getting access token...");
    const tokenResponse = await axios.post(
      `${API_BASE_URL}/v1/oauth2/token`,
      {
        client_id: CLIENT_ID,
        client_secret: CLIENT_SECRET,
      },
      {
        headers: {
          "Content-Type": "application/json",
        },
      }
    );

    const accessToken = tokenResponse.data.access_token;
    console.log("Access token obtained successfully");
    console.log("Token expires in:", tokenResponse.data.expires_in, "seconds\n");

    // Step 2: Create Video with Scene-Level Background Music Control
    console.log("Step 2: Creating video with scene-level background music...");
    const storyboardResponse = await axios.post(
      `${API_BASE_URL}/v2/video/storyboard/render`,
      {
        videoName: "text_to_video_scene_music",
        // Video-level background music (enabled by default for all scenes)
        backgroundMusic: {
          enabled: true,
          autoMusic: true, // Let AI select appropriate music
          volume: 0.3,
        },
        scenes: [
          {
            story: STORY_TEXT_1,
            createSceneOnNewLine: false,
            createSceneOnEndOfSentence: false,
            // This scene uses the video-level background music
          },
          {
            story: STORY_TEXT_2,
            createSceneOnNewLine: false,
            createSceneOnEndOfSentence: false,
            // Disable background music for this specific scene
            backgroundMusic: {
              enabled: false,
            },
          },
          {
            story: STORY_TEXT_3,
            createSceneOnNewLine: false,
            createSceneOnEndOfSentence: false,
            // This scene uses the video-level background music again
          },
        ],
      },
      {
        headers: {
          "Content-Type": "application/json",
          Authorization: accessToken,
        },
      }
    );

    const renderJobId = storyboardResponse.data.data.jobId;
    console.log("Video with scene-level background music render job created");
    console.log("Job ID:", renderJobId, "\n");

    // Step 3: Monitor Job Status
    console.log("Step 3: Monitoring job status...");
    let jobCompleted = false;
    let jobResult = null;

    while (!jobCompleted) {
      const jobStatusResponse = await axios.get(`${API_BASE_URL}/v1/jobs/${renderJobId}`, {
        headers: {
          Authorization: accessToken,
        },
      });

      const status = jobStatusResponse.data.data.status;
      console.log("Current status:", status);

      if (status === "completed") {
        jobCompleted = true;
        jobResult = jobStatusResponse.data;
        console.log("\nVideo with scene-level background music created successfully!");
        console.log("Video URL:", jobResult.data.videoURL);
      } else if (status === "failed") {
        throw new Error("Job failed: " + JSON.stringify(jobStatusResponse.data));
      } else {
        // Wait 5 seconds before checking again
        await new Promise(resolve => setTimeout(resolve, 5000));
      }
    }

    return jobResult;
  } catch (error) {
    console.error("Error:", error.response?.data || error.message);
    throw error;
  }
}

// Run the function
createTextToVideoWithSceneBackgroundMusic();

Python Example

Prerequisites

pip install requests

Complete Code

import requests
import time
import json

API_BASE_URL = 'https://api.pictory.ai/pictoryapis'
CLIENT_ID = 'YOUR_CLIENT_ID'
CLIENT_SECRET = 'YOUR_CLIENT_SECRET'

STORY_TEXT_1 = "AI is poised to significantly impact educators and course creators on social media."
STORY_TEXT_2 = "By automating tasks like content generation, visual design, and video editing, AI will save time and enhance consistency."
STORY_TEXT_3 = "This allows creators to focus on higher-level strategies and ensures a cohesive brand presence."

def create_text_to_video_with_scene_background_music():
    try:
        # Step 1: Get Access Token
        print('Step 1: Getting access token...')
        token_response = requests.post(
            f'{API_BASE_URL}/v1/oauth2/token',
            json={
                'client_id': CLIENT_ID,
                'client_secret': CLIENT_SECRET
            },
            headers={
                'Content-Type': 'application/json'
            }
        )
        token_response.raise_for_status()

        access_token = token_response.json()['access_token']
        print('Access token obtained successfully')
        print(f"Token expires in: {token_response.json()['expires_in']} seconds\n")

        # Step 2: Create Video with Scene-Level Background Music Control
        print('Step 2: Creating video with scene-level background music...')
        storyboard_response = requests.post(
            f'{API_BASE_URL}/v2/video/storyboard/render',
            json={
                'videoName': 'text_to_video_scene_music',
                # Video-level background music (enabled by default for all scenes)
                'backgroundMusic': {
                    'enabled': True,
                    'autoMusic': True,  # Let AI select appropriate music
                    'volume': 0.3
                },
                'scenes': [
                    {
                        'story': STORY_TEXT_1,
                        'createSceneOnNewLine': False,
                        'createSceneOnEndOfSentence': False
                        # This scene uses the video-level background music
                    },
                    {
                        'story': STORY_TEXT_2,
                        'createSceneOnNewLine': False,
                        'createSceneOnEndOfSentence': False,
                        # Disable background music for this specific scene
                        'backgroundMusic': {
                            'enabled': False
                        }
                    },
                    {
                        'story': STORY_TEXT_3,
                        'createSceneOnNewLine': False,
                        'createSceneOnEndOfSentence': False
                        # This scene uses the video-level background music again
                    }
                ]
            },
            headers={
                'Content-Type': 'application/json',
                'Authorization': access_token
            }
        )
        storyboard_response.raise_for_status()

        render_job_id = storyboard_response.json()['data']['jobId']
        print('Video with scene-level background music render job created')
        print(f'Job ID: {render_job_id}\n')

        # Step 3: Monitor Job Status
        print('Step 3: Monitoring job status...')
        job_completed = False
        job_result = None

        while not job_completed:
            job_status_response = requests.get(
                f'{API_BASE_URL}/v1/jobs/{render_job_id}',
                headers={
                    'Authorization': access_token
                }
            )
            job_status_response.raise_for_status()

            status = job_status_response.json()['data']['status']
            print(f'Current status: {status}')

            if status == 'completed':
                job_completed = True
                job_result = job_status_response.json()
                print('\nVideo with scene-level background music created successfully!')
                print(f"Video URL: {job_result['data']['videoURL']}")
            elif status == 'failed':
                raise Exception(f"Job failed: {json.dumps(job_status_response.json())}")
            else:
                # Wait 5 seconds before checking again
                time.sleep(5)

        return job_result

    except requests.exceptions.RequestException as error:
        print(f'Error: {error}')
        if hasattr(error, 'response') and error.response is not None:
            print(f'Response: {error.response.text}')
        raise

# Run the function
if __name__ == '__main__':
    create_text_to_video_with_scene_background_music()

Key Parameters

Video-Level Background Music

backgroundMusic: Configuration applied to all scenes by default
- enabled: true to enable music for all scenes
- autoMusic: true to let AI select music
- musicUrl: URL to custom music file
- volume: Volume level (0 to 1)

Scene-Level Background Music Control

scenes[].backgroundMusic: Override video-level setting for specific scene
- enabled: true or false to enable/disable music for this scene only

How It Works

Video-Level Music: When you set backgroundMusic at the video level, it applies to all scenes by default
Scene-Level Override: You can override the video-level setting for specific scenes
Enable/Disable Only: At the scene level, you can only enable or disable music (not change the music itself)

Use Cases

Disable Music for Specific Scenes

Silent dramatic moments
Important announcements without distraction
Scenes with important dialogue
Testimonials or quotes
Intro/outro scenes with their own audio

Enable Music for Specific Scenes

Adding energy to certain segments
Background ambiance for specific content
Transitions between topics
Emotional emphasis

Common Patterns

Pattern 1: Music on Most Scenes, Off for Key Moments

backgroundMusic: {
  enabled: true,
  autoMusic: true,
  volume: 0.3
},
scenes: [
  { story: "Introduction" },                    // Music on
  { story: "Key point", backgroundMusic: { enabled: false } },  // Music off
  { story: "Supporting details" },              // Music on
  { story: "Call to action", backgroundMusic: { enabled: false } }  // Music off
]

Pattern 2: Music Only on Specific Scenes

backgroundMusic: {
  enabled: false  // Off by default
},
scenes: [
  { story: "Intro", backgroundMusic: { enabled: true } },  // Music on
  { story: "Content" },                                     // Music off
  { story: "Outro", backgroundMusic: { enabled: true } }   // Music on
]

Important Notes

Limited Scene Control: At the scene level, you can only enable/disable music, not change other properties like volume or musicUrl
Same Music: All scenes with music enabled use the same music track (defined at video level)

Response

The API returns a job ID for monitoring the video creation progress. Once completed, you'll receive a video URL with background music applied according to your scene-level configuration.

Notes

Replace YOUR_CLIENT_ID and YOUR_CLIENT_SECRET with your actual API credentials
Scene-level control is limited to enabling/disabling only
Music fades smoothly between scenes
Scene without music may feel more impactful for important content
Background music volume should be balanced with voice-over if present