Performance Tuning with Perl

If you search Google for „perl performance tuning”, you will find quite a few interesting hits. As I had to modify one of the Perl scripts I thought „what the hack, I will try to improve it’s performance too”. Every seasoned programmer will tell you, that this is a dangerous thought. Nevertheless, after I modified the script according to requirements, I took on it’s performance.

Every text I read stressed the fact, that you should test the performance of your script and of the modifications you made. So I familiarized myself with the Benchmark module and ran an initial benchmark:

283 wallclock secs (277.34 usr + 0.67 sys = 278.02 CPU) @ 0.02/s (n=5)

As the code basically reads a log-file from the disk, parses every line and writes out some stuff, I thought I could speed it up by reading the file into memory up front and iterate through that, thus decreasing disk I/O and speeding up the script. Well, the outcome of this „improvement” left me underwhelmed:

256 wallclock secs (252.03 usr + 0.00 sys = 252.03 CPU) @ 0.02/s (n=5)

Not really the improvement I was looking for. So I implemented one of the suggestions, precompiling the regular expressions I am using. As I have arrays of regexes in my script, this sounded promising. I little side-benchmark showed an improvement from 15,4 down to 4 seconds for 100.000 iterations of my little test-script. So I implemented that in my real-word example and got:

258 wallclock secs (253.52 usr + 0.00 sys = 253.52 CPU) @ 0.02/s (n=5)

OK, that was not really nice. So I tried the next tip – „Don’t modify stack”

The following sin is frequently found even in the Perl doc:
my $self = shift;
Unless you have a pertinent reason for this, use this:
my( $self, $x, $y, @z ) = @_;

So I checked my code for „shift @_” and improved that. And this is what I got:

95 wallclock secs (93.09 usr + 0.00 sys = 93.09 CPU) @ 0.05/s (n=5)

WOW. Now that is NICE. From 258 seconds down to 90. That’s what I call an improvement. Now there was one more thing to try – „pass by reference” instead of „pass by value”. As I pass around a lot of text, this actually made sense to me. So I modified all function calls to use „pass by reference” and benchmarked it:

95 wallclock secs (93.94 usr + 0.00 sys = 93.94 CPU) @ 0.05/s (n=5)

OK, that didn’t help as much as I had anticipated. But I am quite happy with the „do not modify stack” improvement. That gave a boost to my script that made a difference. I was surprised that the read ahead of the data did not gave more of a speed boost. I attribute that to the read ahead cache in the modern hard disk drives and operating systems.

My advice: Measure your „improvements”, try a few different approaches and do not rely entirely on synthetic benchmarks. If you can, try to benchmark production code.

Tagged as: , , , , | Author:
[Freitag, 20070105, 20:57 | permanent link | 0 Kommentar(e)

Comments are closed for this story.


Disclaimer

„Leyrers Online Pamphlet“ ist die persönliche Website von mir, Martin Leyrer. Die hier veröffentlichten Beiträge spiegeln meine Ideen, Interessen, meinen Humor und fallweise auch mein Leben wider.
The postings on this site are my own and do not represent the positions, strategies or opinions of any former, current or future employer of mine.
Impressum / Offenlegung gemäß § 25 Mediengesetz

Search

Me, Elsewhere

Tag Cloud

2007, 2blog, 2do, 2read, a-trust, a.trust, a1, accessability, acta, advent, age, ai, amazon, ankündigung, apache, apple, at, audio, austria, backup, barcamp, basteln, bba, big brother awards, birthday, blog, blogging, book, books, browser, Browser_-_Firefox, bruce sterling, buch, bürgerkarte, cars, cartoon, ccc, cfp, christmas, cloud, coding, collection, command line, commandline, computer, computing, concert, conference, copyright, covid19, css, database, date, datenschutz, debian, delicious, demokratie, design, desktop, deutsch, deutschland, dev, developer, development, devops, digitalisierung, digitalks, dilbert, disobay, dna, dns, Doctor Who, documentation, Domino, domino, Douglas Adams, download, downloads, drm, dsk, dvd, e-card, e-government, e-mail, e-voting, E71, education, Ein_Tag_im_Leben, elga, email, encryption, essen, EU, eu, event, events, exchange, Extensions, fail, fedora, feedback, film, firefox, flash, flightexpress, food, foto, fsfe, fun, future, games, gaming, geek, geld, git, gleichberechtigung, google, graz, grüne, grüninnen, hack, hacker, handtuch, handy, hardware, HHGTTG, history, how-to, howto, hp, html, humor, IBM, ibm, ical, iCalendar, image, innovation, intel, internet, internet explorer, iot, iphone, ipod, isp, IT, it, itfails, itfailsAT, itfailsDE, java, javascript, job, jobmarket, journalismus, keyboard, knowledge, konzert, language, laptop, law, lego, lenovo, life, links, linux, Linux, linuxwochen, linuxwochenende, live, living, lol, london, lost+found, lotus, Lotus, lotus notes, Lotus Notes, LotusNotes, lotusnotes, Lotusphere, lotusphere, Lotusphere2006, lotusphere2007, lotusphere2008, Lotusphere2008, lustig, m3_bei_der_Arbeit, mac, mail, marketing, mathematik, media, medien, metalab, Microsoft, microsoft, mITtendrin, mobile, mood, motivation, movie, mp3, multimedia, music, musik, männer, nasa, nerd, netwatcher, network, netzpolitik, news, nokia, Notes, notes, Notes+Domino, office, online, OOXML, open source, openoffice, opensource, orf, orlando, os, outlook, patents, pc, pdf, performance, perl, personal, php, picture, pictures, podcast, politics, politik, pr, press, presse, privacy, privatsphäre, productivity, programming, protest, public speaking, qtalk, quintessenz, quote, quotes, radio, rant, recherche, recht, release, review, rezension, rip, rss, science, search, security, server, settings, sf, shaarli, Show-n-tell thursday, sicherheit, silverlight, smtp, SnTT, social media, software, sony, sound, space, spam, sprache, spö, ssh, ssl, standards, storage, story, stupid, summerspecial, sun, surveillance, sysadmin, talk, talks, technology, The Hitchhikers Guide to the Galaxy, theme, think, thinkpad, thunderbird, tip, tipp, tools, topgear, torrent, towel, Towel Day, TowelDay, travel, truth, tv, twitter, ubuntu, ui, uk, unix, update, usa, usb, vds, video, videoüberwachung, vienna, vim, Vim, vintage, vista, vorratsdatenspeicherung, vortrag, wahl, wcm, web, web 2.0, web2.0, web20, Web20, webdesign, werbung, wien, wiener linien, wikileaks, windows, windows 7, wired, wishlist, wissen, Wissen_ist_Macht, wlan, work, workshops, wow, writing, wtf, wunschzettel, Wunschzettel, www, xbox, xml, xp, zensur, zukunft, zune, österreich, övp, übersetzung, überwachung

AFK Readinglist