Skip to main content

How to Monitor Sync Progress

When importing large Vimeo collections (100+ videos), monitoring progress helps you understand how much time remains and identify any issues before they cause failures. This guide shows you how to effectively track sync jobs.

What You'll Learn​

  • How to interpret progress bar metrics and estimates
  • Techniques for monitoring long-running jobs (30+ minutes)
  • How to identify when a job is stuck vs. legitimately slow
  • How to monitor multiple concurrent jobs
  • How to access detailed progress logs

Prerequisites​

  • Dashboard Access: https://vimeo.trustedgpt.io
  • An In-Progress Sync Job: Start a sync before following this guide, or use the techniques below to monitor future jobs

Understanding Progress Metrics​

Progress Bar Breakdown​

When a sync job is running, the Progress Card displays several metrics:

Visual Progress Bar:

  • Horizontal bar that fills from left to right
  • Gradient color (Vimeo blue → CustomGPT purple)
  • Smooth animation as it updates every 1.5 seconds

Percentage Complete:

  • Displayed on the right side of the progress bar
  • Example: "45%"
  • Calculation: (processed_videos / total_videos) * 100

Processed Count:

  • Displayed on the left side of the progress bar
  • Example: "Processing 45 of 100 videos"
  • Shows exact numbers behind the percentage

Current Video:

  • Displayed below the progress bar
  • Example: "Current: Introduction to Product Features - Q3 2024 Update"
  • Shows which specific video is being processed right now

What Each Metric Tells You​

Percentage Complete → Time Remaining Estimate

While the connector doesn't display an explicit "time remaining" estimate, you can calculate it yourself:

time_remaining = (total_time_elapsed / percentage_complete) * (100 - percentage_complete)

Example:

  • Job has been running for 5 minutes
  • Progress shows 25% complete
  • Estimated total time: 5 / 0.25 = 20 minutes
  • Estimated time remaining: 20 - 5 = 15 minutes

Important: This estimate assumes linear progress. In reality, videos with long transcripts take longer, so progress may not be perfectly linear.

Current Video → Identifying Slowdowns

If the "Current Video" display shows the same video for more than 30 seconds, it suggests:

  • Long Transcript: This video has a lengthy transcript (30+ minutes of content) that takes time to download and parse
  • Network Slowdown: Vimeo's API is responding slowly
  • Rate Limiting: The connector is paused due to hitting rate limits (600 calls/10min)

Processed Count → Absolute Progress

Percentage can be misleading if you don't know the total. Processed count gives you concrete numbers:

  • "Processing 5 of 500 videos" → You're 1% done, expect a long wait
  • "Processing 495 of 500 videos" → Almost finished, just a few videos left

Monitoring Techniques​

Technique 1: Active Monitoring (For Short Jobs)​

Best For: Jobs that complete in under 10 minutes

Steps:

  1. Start the sync job
  2. Keep the dashboard tab open and visible
  3. Watch the progress bar and "Current Video" display
  4. When status changes to "Completed", review results

Advantages:

  • Immediate awareness if something goes wrong
  • You can cancel the job instantly if needed

Disadvantages:

  • Requires you to stay at your computer
  • Not practical for long jobs (30+ minutes)

Technique 2: Passive Monitoring (For Long Jobs)​

Best For: Jobs that complete in 10-60 minutes

Steps:

  1. Start the sync job
  2. Note the start time and approximate duration estimate
  3. Keep the browser tab open (you can switch to other tabs)
  4. Set a timer or reminder for when the job should be done
  5. Return to the dashboard tab to review results

Why Keep Tab Open? The dashboard polls for progress updates every 1.5 seconds. If you close the tab, polling stops. However, the job continues running on the server, so you can always reopen the dashboard later and check the "Recent Jobs" sidebar to see results.

Advantages:

  • You can work on other tasks while the job runs
  • No need to actively watch the progress bar

Disadvantages:

  • You won't know if the job fails until you check back

Technique 3: Background Monitoring (For Very Long Jobs)​

Best For: Jobs that complete in 60+ minutes

Steps:

  1. Start the sync job
  2. Close the dashboard tab (or close your browser entirely)
  3. The job continues running on the server
  4. Return hours later and reopen the dashboard
  5. Check the "Recent Jobs" sidebar for the completed job
  6. Click the job card to view results

How It Works:

  • The backend server runs sync jobs independently of the dashboard
  • Jobs are queued and processed by a background worker
  • Closing the dashboard doesn't cancel the job—it only stops UI updates

Advantages:

  • You don't need to keep a browser tab open for hours
  • No impact on your computer's performance or battery

Disadvantages:

  • No real-time progress updates
  • If the job fails, you won't know until you check back

Technique 4: Multi-Tab Monitoring (For Concurrent Jobs)​

Best For: Running multiple syncs simultaneously

Steps:

  1. Open the dashboard in multiple browser tabs (Cmd/Ctrl + T)
  2. Start a different sync job in each tab
  3. Switch between tabs to monitor each job's progress
  4. Alternatively, tile the tabs side-by-side using your OS window manager

How It Works:

  • Each tab independently polls the backend for job status
  • Jobs run concurrently on the server (limited by backend worker capacity)
  • The "Recent Jobs" sidebar in any tab shows all jobs (not just jobs started in that tab)

Advantages:

  • Monitor multiple jobs from a single browser window
  • Easily compare progress across jobs

Disadvantages:

  • More browser memory usage (one tab per job)
  • Can be confusing to track which tab corresponds to which job

Technique 5: API Monitoring (For Advanced Users)​

Best For: Automated monitoring, scripting, or when you need programmatic access to progress

Steps:

  1. Start a sync job via the dashboard or API
  2. Note the job ID (visible in the URL or API response): sync_a1b2c3d4
  3. Use curl or a script to poll job status:
curl https://vimeo.trustedgpt.io/api/v1/sync/sync_a1b2c3d4
  1. Parse the JSON response to check progress:
{
  "job_id": "sync_a1b2c3d4",
  "status": "running",
  "progress": {
    "processed": 45,
    "total": 100,
    "current_video": "Introduction to Product Features"
  }
}
  1. Repeat the curl request every 5-10 seconds until status changes to completed

Advantages:

  • Fully automated—no need to watch a browser
  • Can integrate with monitoring tools (Nagios, Datadog, etc.)
  • Scriptable for bulk operations

Disadvantages:

  • Requires command-line knowledge
  • No visual UI (text-only output)

See REST API Reference for full API documentation.

Identifying Issues​

Issue: Progress Bar Is Stuck​

Symptoms:

  • Percentage hasn't changed in 5+ minutes
  • "Current Video" shows the same video for 2+ minutes

Possible Causes:

  1. Long Transcript: The current video has a very long transcript (e.g., 2-hour conference talk)
  2. Rate Limiting: The connector is paused due to hitting Vimeo's 600 calls/10min limit
  3. Network Issues: Backend can't reach Vimeo's API
  4. Backend Overload: Backend server is under heavy load and processing slowly

How to Diagnose:

Check Network Status:

  • Look at the connection status indicator in the top-right corner
  • If it shows "Backend not reachable" (red), there's a network issue

Wait 2-3 Minutes:

  • If the progress bar updates after 2-3 minutes, it was likely a long transcript or temporary rate limit pause
  • If it still doesn't update after 5 minutes, proceed to next step

Refresh the Dashboard:

  • Press Cmd/Ctrl + R to refresh the page
  • Check if progress has updated (the job continues running even while you refresh)
  • If progress still shows the same percentage, the job may be stuck

Check Recent Jobs Sidebar:

  • Look for the job in the "Recent Jobs" sidebar
  • If it shows "Running" status with updated progress, the main Progress Card polling might have failed (but the job is fine)

What to Do:

If Job Is Truly Stuck (No Progress for 10+ Minutes):

  1. Cancel the job using the Cancel button
  2. Wait 2-3 minutes for the cancellation to process
  3. Retry the import with the same URL and mode

If Job Is Just Slow (Progress Updates Every 2-5 Minutes):

  • Be patient—large showcases with long transcripts can take 30-60 minutes
  • Consider enabling Token Pool for 6x faster imports (see How to Set Up Token Pool)

Issue: Progress Updates Are Inconsistent​

Symptoms:

  • Progress bar jumps from 10% → 25% → 30% → 45% (skipping percentages)
  • "Current Video" changes too quickly (multiple videos per second) or too slowly (same video for minutes)

Cause: This is normal behavior. Video processing time varies widely:

  • Videos without transcripts: Process in under 1 second (just metadata fetch)
  • Videos with short transcripts: Process in 2-5 seconds
  • Videos with long transcripts (60+ min): Process in 10-30 seconds

What to Do: Nothing. This is expected and doesn't indicate a problem.

Issue: Job Completes But Results Don't Match Expectations​

Symptoms:

  • You imported a showcase with 100 videos, but results show "Added: 50"
  • You expected all videos to have transcripts, but only 10 do

Possible Causes:

  1. Incremental Sync on Repeat Import: If you've imported this URL before, unchanged videos won't be re-added (they'll appear as "Unchanged: 50")
  2. Missing Transcripts: Not all videos have native Vimeo transcripts (see Understanding Transcript Coverage)
  3. Private Videos: Some videos in the collection may be private and not accessible via the API

How to Diagnose:

Check Sync Mode:

  • If you used Incremental Sync, look at the "Unchanged" count
  • Total videos processed = Added + Updated + Unchanged + Deleted

Review Transcript Availability:

  • Open a few videos from the collection on Vimeo.com
  • Check if they have CC (closed captions) buttons in the player
  • If no CC button, those videos don't have native transcripts

What to Do:

If You Want All Videos (Including Unchanged):

  • Re-run the import using Full Sync mode
  • Results will show all videos, including those marked as "Unchanged"

If Transcripts Are Missing:

  • Accept that not all videos have transcripts (metadata is still imported)
  • Contact the video owner to request they upload transcripts
  • Generate transcripts externally and upload to Vimeo

Monitoring Multiple Jobs​

Using the "Recent Jobs" Sidebar​

The "Recent Jobs" sidebar shows all jobs from the past 24 hours, including:

  • Jobs started in the current browser session
  • Jobs started in other browser tabs or windows
  • Jobs started via the API

Key Information Per Job:

  • URL: The Vimeo URL being synced (truncated with "...")
  • Status Badge: "Running", "Completed", "Failed", or "Cancelled"
  • Progress Indicator (for running jobs): Small progress bar showing percentage
  • Timestamp: When the job started (e.g., "5 minutes ago")

How to Use:

  1. Glance at the sidebar to see all active jobs at once
  2. Click any job card to view its details in the main Progress Card
  3. Use the sidebar to compare progress across multiple concurrent jobs

Monitoring from Multiple Tabs​

Scenario: You're importing three showcases simultaneously using three browser tabs.

Steps:

  1. Open Tab 1, start Sync Job A
  2. Open Tab 2, start Sync Job B
  3. Open Tab 3, start Sync Job C
  4. Arrange tabs side-by-side using your OS window tiling feature
  5. Monitor all three progress bars simultaneously

Advantages:

  • Visual comparison of progress across jobs
  • Quickly identify if one job is slower than others

Disadvantages:

  • High browser memory usage
  • Requires large monitor or multiple monitors for effective viewing

Best Practices​

Set Realistic Expectations​

Small Showcases (5-20 videos): 1-3 minutes

Medium Showcases (20-100 videos): 5-15 minutes

Large Showcases (100-500 videos): 20-60 minutes

Very Large Collections (500+ videos): 1-3 hours

With Token Pool Enabled: Divide estimated time by 5-6x (e.g., 60 min → 10 min)

Enable Token Pool for Large Jobs​

If you're regularly importing collections with 100+ videos, set up the Token Pool feature for 6x faster throughput. See How to Set Up Token Pool.

Impact on Monitoring:

  • Progress updates remain the same (every 1.5 seconds)
  • Jobs complete 5-6x faster
  • Less chance of hitting rate limits

Don't Panic If Progress Seems Slow​

It's normal for:

  • Progress bar to pause for 30-60 seconds (processing a video with a long transcript)
  • "Current Video" to stay the same for 30+ seconds (same reason)
  • Progress to jump quickly through some videos and slow down for others (transcript length varies)

Only intervene if:

  • No progress for 10+ minutes
  • Connection status shows "Backend not reachable"
  • Progress bar is stuck at 0% for 5+ minutes

Use Background Monitoring for Very Long Jobs​

If a job will take 60+ minutes, don't keep the browser tab open for the entire duration:

  1. Start the job
  2. Note the approximate completion time
  3. Close the tab (job continues on server)
  4. Return later to check results in "Recent Jobs" sidebar

View Detailed Results:

Optimize for Speed:

Handle Issues:

Understand the System: