Text to Video with Background Music

This example demonstrates how to create a video with background music. You can either let the AI select appropriate music automatically or provide your own custom music file.

Overview

This example covers:

  • Getting an access token
  • Adding custom background music from a URL
  • Using auto-selected music
  • Controlling music volume
  • Trimming music clips
  • Monitoring job status and retrieving the final video

Node.js Example

Prerequisites

npm install axios

Complete Code

import axios from "axios";

const API_BASE_URL = "https://api.pictory.ai/pictoryapis";
const CLIENT_ID = "YOUR_CLIENT_ID";
const CLIENT_SECRET = "YOUR_CLIENT_SECRET";

const STORY_TEXT =
  "AI is poised to significantly impact educators and course creators on social media. By automating tasks like content generation, visual design, and video editing, AI will save time and enhance consistency.";

async function createTextToVideoWithBackgroundMusic() {
  try {
    // Step 1: Get Access Token
    console.log("Step 1: Getting access token...");
    const tokenResponse = await axios.post(
      `${API_BASE_URL}/v1/oauth2/token`,
      {
        client_id: CLIENT_ID,
        client_secret: CLIENT_SECRET,
      },
      {
        headers: {
          "Content-Type": "application/json",
        },
      }
    );

    const accessToken = tokenResponse.data.access_token;
    console.log("Access token obtained successfully");
    console.log("Token expires in:", tokenResponse.data.expires_in, "seconds\n");

    // Step 2: Create Video with Custom Background Music
    console.log("Step 2: Creating video with background music...");
    const storyboardResponse = await axios.post(
      `${API_BASE_URL}/v2/video/storyboard/render`,
      {
        videoName: "text_to_video_with_music",
        backgroundMusic: {
          enabled: true,
          musicUrl: "https://example.com/your-music-file.mp3", // Your custom music URL
          volume: 0.3, // Volume level from 0 to 1 (0.3 = 30%)
          clips: [
            {
              start: 10, // Start at 10 seconds in the music file
              end: 70, // End at 70 seconds in the music file
            },
          ],
        },
        scenes: [
          {
            story: STORY_TEXT,
            createSceneOnNewLine: true,
            createSceneOnEndOfSentence: true,
          },
        ],
      },
      {
        headers: {
          "Content-Type": "application/json",
          Authorization: accessToken,
        },
      }
    );

    const renderJobId = storyboardResponse.data.data.jobId;
    console.log("Video with background music render job created");
    console.log("Job ID:", renderJobId, "\n");

    // Step 3: Monitor Job Status
    console.log("Step 3: Monitoring job status...");
    let jobCompleted = false;
    let jobResult = null;

    while (!jobCompleted) {
      const jobStatusResponse = await axios.get(`${API_BASE_URL}/v1/jobs/${renderJobId}`, {
        headers: {
          Authorization: accessToken,
        },
      });

      const status = jobStatusResponse.data.data.status;
      console.log("Current status:", status);

      if (status === "completed") {
        jobCompleted = true;
        jobResult = jobStatusResponse.data;
        console.log("\nVideo with background music created successfully!");
        console.log("Video URL:", jobResult.data.videoURL);
      } else if (status === "failed") {
        throw new Error("Job failed: " + JSON.stringify(jobStatusResponse.data));
      } else {
        // Wait 5 seconds before checking again
        await new Promise(resolve => setTimeout(resolve, 5000));
      }
    }

    return jobResult;
  } catch (error) {
    console.error("Error:", error.response?.data || error.message);
    throw error;
  }
}

// Run the function
createTextToVideoWithBackgroundMusic();

Python Example

Prerequisites

pip install requests

Complete Code

import requests
import time
import json

API_BASE_URL = 'https://api.pictory.ai/pictoryapis'
CLIENT_ID = 'YOUR_CLIENT_ID'
CLIENT_SECRET = 'YOUR_CLIENT_SECRET'

STORY_TEXT = "AI is poised to significantly impact educators and course creators on social media. By automating tasks like content generation, visual design, and video editing, AI will save time and enhance consistency."

def create_text_to_video_with_background_music():
    try:
        # Step 1: Get Access Token
        print('Step 1: Getting access token...')
        token_response = requests.post(
            f'{API_BASE_URL}/v1/oauth2/token',
            json={
                'client_id': CLIENT_ID,
                'client_secret': CLIENT_SECRET
            },
            headers={
                'Content-Type': 'application/json'
            }
        )
        token_response.raise_for_status()

        access_token = token_response.json()['access_token']
        print('Access token obtained successfully')
        print(f"Token expires in: {token_response.json()['expires_in']} seconds\n")

        # Step 2: Create Video with Custom Background Music
        print('Step 2: Creating video with background music...')
        storyboard_response = requests.post(
            f'{API_BASE_URL}/v2/video/storyboard/render',
            json={
                'videoName': 'text_to_video_with_music',
                'backgroundMusic': {
                    'enabled': True,
                    'musicUrl': 'https://example.com/your-music-file.mp3',  # Your custom music URL
                    'volume': 0.3,  # Volume level from 0 to 1 (0.3 = 30%)
                    'clips': [
                        {
                            'start': 10,   # Start at 10 seconds in the music file
                            'end': 70      # End at 70 seconds in the music file
                        }
                    ]
                },
                'scenes': [
                    {
                        'story': STORY_TEXT,
                        'createSceneOnNewLine': True,
                        'createSceneOnEndOfSentence': True
                    }
                ]
            },
            headers={
                'Content-Type': 'application/json',
                'Authorization': access_token
            }
        )
        storyboard_response.raise_for_status()

        render_job_id = storyboard_response.json()['data']['jobId']
        print('Video with background music render job created')
        print(f'Job ID: {render_job_id}\n')

        # Step 3: Monitor Job Status
        print('Step 3: Monitoring job status...')
        job_completed = False
        job_result = None

        while not job_completed:
            job_status_response = requests.get(
                f'{API_BASE_URL}/v1/jobs/{render_job_id}',
                headers={
                    'Authorization': access_token
                }
            )
            job_status_response.raise_for_status()

            status = job_status_response.json()['data']['status']
            print(f'Current status: {status}')

            if status == 'completed':
                job_completed = True
                job_result = job_status_response.json()
                print('\nVideo with background music created successfully!')
                print(f"Video URL: {job_result['data']['videoURL']}")
            elif status == 'failed':
                raise Exception(f"Job failed: {json.dumps(job_status_response.json())}")
            else:
                # Wait 5 seconds before checking again
                time.sleep(5)

        return job_result

    except requests.exceptions.RequestException as error:
        print(f'Error: {error}')
        if hasattr(error, 'response') and error.response is not None:
            print(f'Response: {error.response.text}')
        raise

# Run the function
if __name__ == '__main__':
    create_text_to_video_with_background_music()

Key Parameters

Background Music Object

  • enabled: Boolean - Set to true to enable background music
  • musicUrl (optional): URL to your custom music file
  • autoMusic (optional): Boolean - Let AI select appropriate music automatically
  • volume (optional): Number between 0 and 1 (default varies)
    • 0 = Muted
    • 0.3 = 30% volume (soft background)
    • 0.5 = 50% volume (balanced)
    • 1 = 100% volume (full volume)
  • clips (optional): Array of clip objects to trim the music (max 10 clips)
    • start: Start time in seconds
    • end: End time in seconds

Using Auto-Selected Music

Instead of providing a custom music URL, you can let the AI select appropriate music:

backgroundMusic: {
  enabled: true,
  autoMusic: true,
  volume: 0.4
}

Important Rules

  1. Cannot Use Both: You cannot provide both musicUrl and autoMusic in the same request
  2. Volume Range: Volume must be between 0 and 1
  3. Clip Limits: Maximum 10 music clips can be specified
  4. Clip Timing: End time must be greater than start time

Multiple Music Clips

You can use multiple segments from the same music file:

clips: [
  { start: 0, end: 30 }, // First 30 seconds
  { start: 60, end: 90 }, // 30 seconds starting from 1 minute
  { start: 120, end: 180 }, // 1 minute starting from 2 minutes
];

Use Cases

  • Add branded music to marketing videos
  • Create mood with background music
  • Enhance educational content with subtle background tracks
  • Match music to video content automatically with autoMusic
  • Use specific portions of longer music tracks

Supported Audio Formats

Common audio formats supported include:

  • MP3
  • WAV
  • AAC
  • M4A

Response

The API returns a job ID for monitoring the video creation progress. Once completed, you'll receive a video URL with the background music mixed at the specified volume.

Notes

  • Replace YOUR_CLIENT_ID and YOUR_CLIENT_SECRET with your actual API credentials
  • Replace the music URL with your actual music file URL
  • The music file must be accessible via a public URL
  • Background music is mixed with voice-over (if present)
  • Lower volume (0.2-0.4) is recommended when voice-over is present
  • Music loops if shorter than the video duration
  • Music fades out at the end of the video