Devel::MemoryTrace::Light Tricks

2011-06-26 15:08:55 UTC

Recently I introduced Devel::MemoryTrace::Light and moments ago I just pushed 0.08 to CPAN which contained a minor bug fix.

I'd like to take a few moments while I'm sitting here at the airport waiting to fly to Asheville, NC for YAPC::NA 2011 to introduce a few nifty things you can do with Devel::MemoryTrace::Light.

Tracing with source output

The default behavior of this module is all well and good, but I thought it'd be useful to have the source in question when memory increases are detected rather than having to always go look through the files. With the following custom provider, this is possible:

Source: example1.pl 

#!/usr/bin/perl

# Print a message when memory increase is detected, include source code of the
# line that caused it

use strict;
use warnings;

BEGIN {
  # Only load up the callback if Devel::MemoryTrace::Light is being used
  if ($Devel::MemoryTrace::Light::VERSION) {

	# Override default output when memory change is detected
	DB::set_callback(
	  sub  {
		my ($pkg, $file, $line, $bytes) = @_;

		print "Package $pkg used $bytes bytes in file $file on line $line: ";

		no strict 'refs';
		print ${"_<$file"}[$line] || "(No source)\n";
		use strict 'refs';
	  }
	);
  }
}

# Example...
my @mem;

$mem[44096] = 1;

$mem[55096] = 2;

If you run the above program, you'll see something like: 

$ perl -d:MemoryTrace::Light ./example1.pl
Package main used 262144 bytes in file example1.pl on line 30: $mem[44096] = 1;
Package main used 540672 bytes in file example1.pl on line 32: $mem[55096] = 2;

Getting a summary of statistics by package/file/line

Perhaps a slightly more interesting use of this module would be to get statistics on the top N memory hogs in a perl program. With the following provider, this is easily done:

Source: example2.pl 

#!/usr/bin/perl

# Print out a report of the top 5 memory users by:
#  - package
#  - file
#  - file/line

use strict;
use warnings;

BEGIN {
  # Only load up the callback if Devel::MemoryTrace::Light is being used
  if ($Devel::MemoryTrace::Light::VERSION) {

	my %stats;

	# Override default output when memory change is detected
	DB::set_callback(
	  sub  {
		my ($pkg, $file, $line, $bytes) = @_;

		$stats{'package'}{$pkg}            += $bytes;
		$stats{'file'}{$file}              += $bytes;
		$stats{'file/line'}{"$file:$line"} += $bytes;
	  }
	);

	END {
	  # END blocks get called regardless of if (); need to check again
	  if ($Devel::MemoryTrace::Light::VERSION) {

		# Don't trace this end block!
		DB::disable_trace();

		for my $key (qw( package file file/line )) {
			my @top = reverse sort {
				$stats{$key}{$a} <=> $stats{$key}{$b}
			} keys %{ $stats{$key} };

			my $max = scalar @top > 5 ? 5 : scalar @top;

			print "Top $max memory users by $key\n";

			for my $idx (0..$max - 1) {
				print "\t$stats{$key}{$top[$idx]}: $top[$idx]\n";
			}
		}

		DB::enable_trace();
	  }
	}
  }
}

my @mem;

$mem[44096] = 1;

$mem[55096] = 2;

Run the above program, and you'll get something like: 

$ perl -d:MemoryTrace::Light ./example2.pl
Top 1 memory users by package
        802816: main
Top 1 memory users by file
        802816: example2.pl
Top 2 memory users by file/line
        540672: example2.pl:59
        262144: example2.pl:57

Happy debugging!

-- Matthew Horsfall (alh)