Cùran's life
A Debian Developer's observations

2nd March 2011 10:05 (GMT)
Downloading entire YouTube playlists

Downloading videos from YouTube isn't a problem thanks to tools like clive. But fetching all videos of a playlist was always a little bit tricky. There is clivescan, but it requires user intervention and needs Tk. Other tools have focused on feed parsing, so in case you have a playlist feed you can then download all (listed) videos. But none of the tools offered me a simple way to download all videos in a playlist from the command line. That's when I started writing ytplaylistfetcher, a simple shell script aiming to parse a YouTube playlist, extract all unique video ids and pass the list on to clive for downloading.

By now ytplaylistfetcher is pretty much complete and there aren't many changes ahead (unless YouTube changes its page layout again, e.g. back to multi-page playlists).

At the moment ytplaylistfetcher can handle the three common playlist link formats. If you've encountered another format or found a playlist link that doesn't work, let me know.

ytplaylistfetcher has just one limitation in my opionin (but one I can live well with): it is limited to YouTube playlists. But that isn't a severe drawback in my opinion, as I rarely receive playlist links to other video hosting platforms (at least none where a simple wget call doesn't solve the problem.

This little script show-cases one reason, why I like Linux so much: there is always an easy way to extend existing functionality.

Permalink | clive, debian.
14th March 2011 13:44 (GMT)
ytplaylistfetcher: new playlist URL format

It seems like YouTube has added a new format for playlist URLs. You'll find it on search result pages. The ID of the playlist comes after:


So a complete URL containing this format might look like http://www.youtube.com/watch?v=hzMtVJTxrJw&playnext=1&list=PL00A3D4ABEAF28FA3

I've added support for this format in bcccc0f7 to YTPlaylistFetcher. You can download the latest version by right-clicking on the previous link and select "Save Link as ..." from the context menu.

Permalink | clive, debian.
5th August 2011 14:12 (GMT)
YouTube: download 1080p videos when CLive is unavailable

Today's addition to the tips series has a funny history. A friend of mine was stuck in an envrionment, where she couldn't install Perl or CLive but PHP with some PECL modules was installed. She needed to download a few videos for a presentation, but didn't know how to fetch the videos from YouTube. So I was asked to help. ;-)

After some pondering of the problem, I came up with the following script:

 * Copyright (C) 2011  Kai Wasserbäch <kai@dev.carbon-project.org>
 * Small little script to help out in the absence of Perl/Clive but with
 * PHP present.
 * License: GPLv3 <http://www.gnu.org/licenses/gpl-3.0.txt>

if(php_sapi_name() !== 'cli')
  die("This script needs to be run on the CLI!\n");

if($argc !== 2 )
  die("No URL given on CLI or too much parameters!\n");

// Script expects a URL like http://www.youtube.com/watch?v=[ID]
if(filter_var($argv[1], FILTER_VALIDATE_URL, FILTER_FLAG_QUERY_REQUIRED) === false)
  die("Parameter given as URL does not seem to be an URL or contain a query!\n");

if(!$ch = curl_init($argv[1]))
  die("Couldn't init cURL handle for »".$argv[1]."«\n");
curl_setopt($ch, CURLOPT_HEADER,          false );
curl_setopt($ch, CURLOPT_RETURNTRANSFER,  true  );
$htmlsource = curl_exec($ch);
$error = curl_error($ch);
if($htmlsource === false)
  die('cURL returned an error: '.$error."\n");

               $htmlsource, $matches))
  die("No matches found for url_encoded_fmt_stream_map!\n");

$final = NULL;
foreach(explode(',', urldecode($matches[1])) as $decoded_url_map) {
  parse_str($decoded_url_map, $tmp);
  if($tmp['quality'] === 'hd1080') {
    $final = $tmp;
    $final['url']   = urldecode($tmp['url']);
    // $final['type']  = urldecode($tmp['type']); // Not used
    break;  // Not really needed, but should save us some
            // iterations, since 1080p seems to come always first.

if(!is_array($final) || is_null($final))
  die("No 1080p version of the video was found!\n");

if(!preg_match('/<meta property="og:title" content="(.*)"/', $htmlsource, $title))
  die("Couldn't determine title!\n");

$ftitle = preg_replace('/[^\w\d]*/', '', $title[1]);
printf("Found hd1080-quality video for »%s«.\n", $argv[1]);
printf("Downloading to %s.mp4 ...", $ftitle);
  die("\nCouldn't create cURL handle for »".$final['url']."«\n");

if(!$fh = fopen(getcwd().DIRECTORY_SEPARATOR.$ftitle.'.mp4', 'wb')) {
  die("\nCouldn't create »".$ftitle.".mp4«.\n");
curl_setopt($ch, CURLOPT_HEADER,         false );
curl_setopt($ch, CURLOPT_FILE,           $fh   );
curl_setopt($ch, CURLOPT_BINARYTRANSFER, true  );
if(!curl_exec($ch)) {
  $error = curl_error($ch);
  die("\nCouldn't download video. cURL error: ".$error."\n");
printf(" done.\n");

You can download it from my website or you can just copy the code given above into a file. No matter how you obtain the code, the invocation will look like: php -f /path/to/script.php http://www.youtube.com/watch?v=[ID].

Now, before I get a lot of e-mails telling me, what could be improved on the script or its other limitations (like only being able to download 1080p videos), please remember that it was hacked together for a simple use case with no real intent on my part to use it for anything else. It only later occurred to me, that others might be in a similar pinch sometime. Thus I'm releasing the above code "as is" under the GNU GPLv3. Therefore you're free to adapt it to your needs. Please also note, that I'm not recommending using the above script instead of CLive. If you can install CLive, do so and don't bother with the above.

Permalink | cheat-sheet, clive, debian, php.

Common Blog License: Creative Commons Attribution-ShareAlike 3.0 Unported License | Imprint (Impressum) | Compiled with Chronicle v4.6


Support my Debian work!