As the local geek I get all sorts of (Linux) questions asked, like "How can you delete the nth line with Sed?" or "Is there a way to search for the following in a file?" (the latter being a request to construct a regular expression for grep). And while I'm pretty sure, you can find answers for such questions quickly with $SEARCH_ENGINE I find myself generally typing the answer into the IM session. This, and just seeing Bernd's post about Vim on Planet Debian, prompted me to start a little, irregular series of posts, to which I can point people, whenever I get asked such questions.

I start this off with a tip for Vim, a very powerful text editor. The problem is simple: you get (source code) files with trailing whitespaces (sometimes accumulated in "empty" lines). This makes diffing (and merging) difficult. Thus the question is: how do I prevent that from happening? How do I notice, that I have whitespaces at the end of a line? The solution consists of two lines in your .vimrc:

:highlight TrailWhitespace ctermbg=red guibg=red
:match TrailWhitespace /\s\+$\| \+\ze\t/

If you just want this functionality if syntax highlighting is also active, then you should use

:autocmd Syntax * syn match TrailWhitespace /\s\+$\| \+\ze\t/

instead of the :match line. The regular expression used in both cases matches trailing whitespaces and whitespaces in front of tabs.

Of course there are several other options on how to do this or what you might want to highlight, but that would be beyond the scope of this little post.

Yesterday's blog entry garnered a lot of attention and some responses (in the form of e-mails, which is totally fine). Two of them deserve, as far as I'm concerend, their own little entry in the mini-series I kicked off. So, here we go:

First up was reader Sylvain "ythier" Hitier, who suggested adding

set list lcs=eol:¶,tab:»-,trail:·

to your .vimrc and making use of the :list option. The above line will highlight tabulators, trailing whitespaces and the end of a line. And it'll allow you to differentiate the three.
A warning though: you would want to deactivate this, in case you want to do copy and pasting with your mouse.

[UPDATE] Sylvain just sent me a little key binding for C&P he's using to toggle the highlighting:

function CutPaste()
  setlocal paste! nu! list!
endfunction
map <F10> :call CutPaste()

[/UPDATE]

The second really nice idea came by courtesy of Thilo Six to my attention:

" automatically delete trailing whitespace & Dos-returns    {{{2
fun! <SID>MyDeleteTrailingWhitespace()
  if ! &bin
    let l:l = line(".")
    let l:c = col(".")
    silent! :%s/[\r \t]\+$//
    call histdel("search", -1)
    call cursor(l:l, l:c)
  endif
endfun
autocmd BufWritePre,FileWritePre * call <SID>MyDeleteTrailingWhitespace()

The above snippet will automatically delete trailing whitespace and carriage returns as added by a lot of Windows programs.
Again a little warning: the function is executed automatically, so be sure you have only replacements included in the regular expression you really want to get deleted!

Thanks a lot for all the input I've received!

From time to time I get e-mails about my wine-unstable packages being uninstallable. Most often it turns out, that the problem stems from using e.g. oldstable (Lenny) instead of at least stable (Squeeze). The same problem arises, if you wish to install the wine-unstable packages on a platform for which I don't offer binaries. In either case you need to build the package from source, which isn't as difficult, as many think.

This entry in my ongoing tips mini-series of posts is thus going to detail how you setup pbuilder and afterwards use it to build a package. Please note, that I'm just focusing on pbuilder here and ignore things like cowbuilder.

Preparations

The following steps are needed just once, in general. Some would be needed again, if you want to create a second base.tgz, e.g. for a different distribution.

1. Install pbuilder with your favourite package manager, e.g. aptitude: aptitude install pbuilder
2. Become root
3. Configure pbuilder. Have a look at /etc/pbuilderrc, /usr/share/doc/pbuilder/examples/pbuilderrc and the manual page for pbuilderrc for reference. If you want to change some settings made in /etc/pbuilderrc, create a /root/.pbuilderrc and override any settings you don't like.
4. Create the base.tgz (the chroot environment for your builds). In the easiest case all your settings are made in the configuration files and you just need to run pbuilder --create, which creates the base.tgz in /var/cache/pbuilder, unless you changed the configuration.
   You should also note, that you can override certain settings, like the distribution or the path to (and name of) the base.tgz file created. See the manual page for pbuilder for all options and when they can be used.

Build a package

The following steps are needed for every build:

1. Download the source package, you want to build. No matter how you fetch them (dget, apt-get source, …), you should end up with three files, one of them ending in .dsc.
2. Become root.
3. Update your base.tgz file: pbuilder --update. Again, see the manual page for further options. (You can skip this step, if you just created the base.tgz or you just updated it for another build.)
4. Build the source package: run pbuilder --build /path/to/source-package.dsc (obviously you need to replace the last argument with a real path)
5. If you haven't changed the configuration of pbuilder in that regard, then your binaries end up in /var/cache/pbuilder/result. Install the binaries, you want, from there (with dpkg -i).

Now you should be able to build your own packages from source!

And to answer one question, that some of you might have now ("Why use pbuilder at all and not just build the package with dpkg-buildpackage directly?"): because pbuilder allows you to reliably build in a clean environment. That can safe you a lot of trouble, especially if you modified the source package. And it helps to keep the number of installed packages on your system down.

Today's addition to the tips series has a funny history. A friend of mine was stuck in an envrionment, where she couldn't install Perl or CLive but PHP with some PECL modules was installed. She needed to download a few videos for a presentation, but didn't know how to fetch the videos from YouTube. So I was asked to help. ;-)

After some pondering of the problem, I came up with the following script:

<?php
/*
 * Copyright (C) 2011  Kai Wasserbäch <kai@dev.carbon-project.org>
 *
 * Small little script to help out in the absence of Perl/Clive but with
 * PHP present.
 *
 * License: GPLv3 <http://www.gnu.org/licenses/gpl-3.0.txt>
 *
 */

if(php_sapi_name() !== 'cli')
  die("This script needs to be run on the CLI!\n");

if($argc !== 2 )
  die("No URL given on CLI or too much parameters!\n");

// Script expects a URL like http://www.youtube.com/watch?v=[ID]
if(filter_var($argv[1], FILTER_VALIDATE_URL, FILTER_FLAG_QUERY_REQUIRED) === false)
  die("Parameter given as URL does not seem to be an URL or contain a query!\n");

if(!$ch = curl_init($argv[1]))
  die("Couldn't init cURL handle for »".$argv[1]."«\n");
curl_setopt($ch, CURLOPT_HEADER,          false );
curl_setopt($ch, CURLOPT_RETURNTRANSFER,  true  );
$htmlsource = curl_exec($ch);
$error = curl_error($ch);
curl_close($ch);
if($htmlsource === false)
  die('cURL returned an error: '.$error."\n");
unset($error);

if(!preg_match('/url_encoded_fmt_stream_map=([a-zA-Z0-9%\.,\-_]+)&/',
               $htmlsource, $matches))
  die("No matches found for url_encoded_fmt_stream_map!\n");

$final = NULL;
foreach(explode(',', urldecode($matches[1])) as $decoded_url_map) {
  parse_str($decoded_url_map, $tmp);
  if($tmp['quality'] === 'hd1080') {
    $final = $tmp;
    $final['url']   = urldecode($tmp['url']);
    // $final['type']  = urldecode($tmp['type']); // Not used
    break;  // Not really needed, but should save us some
            // iterations, since 1080p seems to come always first.
  }
}

if(!is_array($final) || is_null($final))
  die("No 1080p version of the video was found!\n");

if(!preg_match('/<meta property="og:title" content="(.*)"/', $htmlsource, $title))
  die("Couldn't determine title!\n");

$ftitle = preg_replace('/[^\w\d]*/', '', $title[1]);
printf("Found hd1080-quality video for »%s«.\n", $argv[1]);
printf("Downloading to %s.mp4 ...", $ftitle);
if(!$ch=curl_init($final['url']))
  die("\nCouldn't create cURL handle for »".$final['url']."«\n");

if(!$fh = fopen(getcwd().DIRECTORY_SEPARATOR.$ftitle.'.mp4', 'wb')) {
  curl_close($ch);
  die("\nCouldn't create »".$ftitle.".mp4«.\n");
}
curl_setopt($ch, CURLOPT_HEADER,         false );
curl_setopt($ch, CURLOPT_FILE,           $fh   );
curl_setopt($ch, CURLOPT_BINARYTRANSFER, true  );
if(!curl_exec($ch)) {
  $error = curl_error($ch);
  fclose($fh);
  curl_close($ch);
  die("\nCouldn't download video. cURL error: ".$error."\n");
}
curl_close($ch);
fclose($fh);
printf(" done.\n");
?>

You can download it from my website or you can just copy the code given above into a file. No matter how you obtain the code, the invocation will look like: php -f /path/to/script.php http://www.youtube.com/watch?v=[ID].

Now, before I get a lot of e-mails telling me, what could be improved on the script or its other limitations (like only being able to download 1080p videos), please remember that it was hacked together for a simple use case with no real intent on my part to use it for anything else. It only later occurred to me, that others might be in a similar pinch sometime. Thus I'm releasing the above code "as is" under the GNU GPLv3. Therefore you're free to adapt it to your needs. Please also note, that I'm not recommending using the above script instead of CLive. If you can install CLive, do so and don't bother with the above.

After a little pause, I thought I add another entry to my ongoing tips series. Today we'll get rid of the black bars you can most likely observe on a widescreen display, when playing a video DVD with MPlayer.

This little post assumes that you've already inserted the DVD you'd like to watch, your MPlayer setup works with dvd:// style addresses, you know which title from the DVD you want to watch (the examples will use the first title) and you know your monitor's aspect ratio (the examples uses 16:9).

Now all that is needed are the following two steps:

1. Detect the cropping parameters:

   mplayer -vf cropdetect -monitoraspect 16:9 dvd://1

   The previous line is followed by the usual MPlayer header and output. The film should start playing and after a few seconds you can press q (just wait until the detected values stop changing). Now there should be a few lines looking like:

   [CROP] Crop area: X: 0..719  Y: 76..499  (-vf crop=720:416:0:80).
   A:   1.5 V:   1.5 A-V: -0.000 ct: -0.018  38/ 38  5%  7%  1.0% 0 0
   [CROP] Crop area: X: 0..719  Y: 75..500  (-vf crop=720:416:0:80).
   A:   1.6 V:   1.6 A-V: -0.000 ct: -0.018  39/ 39  5%  7%  1.1% 0 0
   [CROP] Crop area: X: 0..719  Y: 75..500  (-vf crop=720:416:0:80).
   A:   1.6 V:   1.6 A-V: -0.000 ct: -0.018  40/ 40  5%  7%  1.1% 0 0

2. Play the film with the detected crop values and the correct aspect for the video material (4:3 for PAL and 3:2 for NTSC):

   mplayer -vf crop=720:416:0:80 -monitoraspect 16:9 -aspect 4:3 dvd://1

Of course you can add any other options to the invocation as usual, though you only need to add those to the second invocation (e.g. select the correct audio stream or start in fullscreen mode). Also, you might want to set the monitor aspect permantly in your MPlayer configuration (just add monitoraspect = "16:9"), as the monitor rarely changes for the average machine /etc/mplayer/mplayer.conf.local is probably the best place (make sure your /etc/mplayer/mplayer.conf includes that file at the end).

Now you should be able to enjoy all video DVDs on your widescreen display without black bars at the top and bottom!

Maybe you've already heard, that Google is starting to charge a fee for using the Maps API (Google may decide you're an non-profit organization and waive the fees). That is, in my opinion, Google telling you to look for better alternatives. And thanks to the awesome effort of the OpenStreetMaps community, there is such an alternative.

Thus today's addition to the mini-tips series will tell you, how you get an OpenStreetMaps-powered map on your website.

1. First you need to determine the coordinates, where you want to center, the map. You can do this with OpenRouteService.org. Just search for the address, right-click on the marker highlighting the result, and set it as e.g. your start point, that'll give you the coordinates in the start field on the left. Alternatively you can just move your pointer over a place on the map and have a look at the lower right corner, where the current coordinates for the pointer position are displayed.
2. Then you need to add a few things to the website, where the card is to be shown:

   • Include the OpenLayers script in the <head> of your webpage:

     <script src="http://www.openlayers.org/api/OpenLayers.js" type="text/javascript" />
     

   • Add a short ECMAScript snippet right after that, which will initialize your map:

     function osmInit() {
       document.getElementById("ol_map").style.display = 'block';
       var lonLat = new OpenLayers.LonLat(13.376096, 52.518598) // Center of the map
       .transform(
         new OpenLayers.Projection("EPSG:4326"),   // transform from WGS 1984
         new OpenLayers.Projection("EPSG:900913")  // to Spherical Mercator Projection
       );
       var map = new OpenLayers.Map("ol_map");
       var osmlayer = new OpenLayers.Layer.OSM();
       map.addLayer(osmlayer);
       map.setCenter(lonLat,
         16 // Zoom level
       );
     }
     

     The first line is not needed, but I find it nice to hide the map on default, in case the user has disabled ECMAScript they haven't a large empty area somewhere on the website. The first line changes then the display value from "none" to "block". The rest is, I hope, already explained by the code itself.

   • Now only two more changes are needed, first you need to add onload="osmInit();" to your <body> tag. And afterwards an element (say a <p>) with the ID ol_map somewhere in the website <body>. You should set a width and height for that tag (style is your friend), otherwise the map might be bigger than you like.
3. Done. Enjoy your free (not just as in free beer) map!

You can, of course, customize your map further by adding additional layers and changing the available options (e.g. the zoom level). If you would want to mark a location on your map, you'd need to add something like

var markers = new OpenLayers.Layer.Markers("POI1");
map.addLayer(markers);
markers.addMarker(new OpenLayers.Marker(lonLat));

to your ECMAScript snipped after you added the OpenStreetMaps layer. These three lines would mark the location given by the variable lonLat.

For further information, you can check out the OpenStreetMaps wiki and the OpenLayers documentation.

In case you wonder, where my coordinates from this example point, you can check. Oh, and don't forget to consider a donation to OpenStreetMaps, so the project can pay the bills for its servers.