Sunday, 6 April 2014

Perl Dancer on Media Temple (gs)

Recently I was asked by a friend to install Dancer on Media Temple Grid Service (gs), and after I couldn't find a tutorial to do so, I decided to write one myself :)

Hopefully, you'll be able to get up an running in seven easy steps. I am assuming you know how to use remote *nix shells via ssh, editing files, etc.

You should be aware that none of this is really supported by Media Temple so if you run into trouble or if something changes on the grid, you're on your own. For anything other than simple dynamic websites (in PHP), Media Temple recommends their more robust (and expensive) dedicated virtual (dv) server solution.


Step #1 - Login with SSH on Media Temple


At the time of this writing, Media Temple's GS runs on Perl 5.10.1, which is not very recent, but is good enough for us. Teaching you how to setup ssh is a bit beyond the scope of this tutorial, but thankfully Media Temple provides a very nice ssh guide.

Media Temple runs Apache, but error logging is not enabled by default, so you should probably enable error_log on Media Temple. If you run into any trouble on the following sections of this tutorial, following the error log (by something like running "tail -f $HOME/../../logs/error_log" on another window) is definitely going to help you!

Step #2 - Sanity check


After you log in, you'll see Media Temple's domains come with 2 directories (at least that's how they showed up to me): "cgi-bin" and "html". This structure implies that dynamic content should be under cgi-bin and static content should be under "html". Just to make sure everything is sane, up and running, let's create a very simple pure-perl script and place it under the "cgi-bin" directory:

    #!/usr/bin/perl
    # ^^^ Media Temple requires the shebang!
    use strict;
    use warnings;
    print "Content-type: text/html\n\n";
    print "Hello, Media Temple";

Save this file as "test.cgi" and set permissions to 700 (required, apparently):

    chmod 700 test.cgi

The owner of the file should be the main account for that domain as well:

    chown $USER:$USER test.cgi

(where $USER is the username of the main account)

You should also make sure the cgi-bin directory's permissions is set to exactly 755, otherwise things won't work.

Now try and run it from your web browser, by opening "http://your-domain/cgi-bin/test.cgi", where "your-domain" is your Media Temple domain. If you see an "Internal Server Error" message, try checking out your error_log and review the previous steps (for example, make sure the script is exactly as it is above, and all permissions are right).

Step #3 - Installing Perl modules on Media Temple (mt) Grid


Media Temple provides a Knowledge Base tutorial on how to install Perl modules on the Grid, but it only lists manual installations which are boring and error prone. After all, we live in the future, where it's simple to install Perl modules as a user on a shared hosting service. All you have to do is type the following on your server's command line (it's just one line, and you can cut and paste as it is):

    curl -L http://cpanmin.us | perl - --local-lib=~/perl5 local::lib App::cpanminus && eval $(perl -I ~/perl5/lib/perl5/ -Mlocal::lib)

 This is going to install local::lib and cpanm into your home directory, and enable you to install any other modules there by simply typing "cpanm My::Module" (where "My::Module" is, of course, the name of the module you want to install).

While you're at it, remember to add the line:

    eval $(perl -I$HOME/perl5/lib/perl5 -Mlocal::lib)

to your ~/.bash_profile file (create it if necessary, of course).

Step #4 - Installing Dancer (and everything else!)


After the previous step, installing Dancer is easy as pie:

    cpanm Dancer

Done! Let's also install Plack::Runner, in case we need it:

   cpanm Plack::Runner

You can take this moment to install all your app's dependencies. For instance, my friend's app is also using Template Toolkit and DBIx::Class

   cpanm Template DBIx::Class

Step #5 - Making a default Dancer app (or moving your current one)


If you already have a Dancer app, just move it to Media Temple's domain directory. For example, if your Dancer app is called "my-app", move it to:  "/home/XXXXX/domains/mydomain.com/my-app" (replacing XXXXX and "mydomain.com" to the appropriate values for your grid account).

Otherwise, just go there (/home/XXXX/domains/mydomain.com) and create your Dancer app by typing "dancer -a appname".


Step #6 - Change "html" to a symlink:


Your Dancer app's base directory is usually "appname/public", but Media Temple forces you to have an "html" directory as base. So we remove the original (just rename it instead if you're not brave enough :) and create it again, only this time as a symbolic link to "appname/public":

    cd ~/domains/mydomain.com
    mv html html_old
    ln -s myapp/public html

Step #7 - Add the appropriate .htacess files


Media Temple runs Apache on the grid server, so we control what runs with .htaccess files. This is the content we put in the file "myapp/public/.htaccess":

    # Allow /public to run Perl code
    Options +ExecCGI +FollowSymLinks
    AddHandler cgi-script .cgi .pl


    # Initialize mod_rewrite
    RewriteEngine On
    RewriteBase /
 

    # If file doesn't exist on the server, dispatch!
    RewriteCond %{REQUEST_FILENAME} !-f
    RewriteRule ^(.+)$ dispatch.cgi/$1 [L]



At this point I bumped into trouble. Media Temple's Apache doesn't seem to let Perl see variables such as PERL5LIB. There's likely a clean way to fiddle with "SetEnv" on .htaccess to fix this (let me know if you get it working!),  but what I did was add a "use lib" line on top of public/dispatch.cgi, like so:

#!/usr/bin/env perl
use 5.10.0;

# Media Temple doesn't read our environment
use lib qw(
   /home/XXXXX/users/.home/perl5/lib/perl5
   /home/XXXXX/users/.home/perl5/lib/perl5/x86_64-linux-gnu-thread-multi
);

That's the basic setup! If everything went well, you should be able to point your web browser to your domain and see Dancer's main page \o/

You should notice we're running Dancer on standard cgi, which means it's going to be somewhat slow, specially when loading modules.

Appendix: Perl with FastCGI on the Media Temple Grid


Sadly, Media Temple doesn't seem to provide proper FastCGI support for Perl - at least from my understanding. Of course, that didn't stop me from trying, so here's what I did:

First, enable PHP FastCGI on the GS. It should have created a .htaccess in your domain root (i.e. the parent of your "html" dir) with some basic FastCGI settings. I changed them to this:

    # BEGIN (mt) controlled settings
    <IfModule !mod_fcgid.c>
      AddHandler cgi-script .cgi .pl
    </IfModule>
    <IfModule mod_fcgid.c>
      AddHandler fcgid-script .fcgi
      <Files *.fcgi>
        Options +ExecCGI
      </Files>
    </IfModule>
    # END (mt) controlled settings


(this is basically their default script, with "php" replaced with "cgi", "pl" and "fcgi").

Then I went to our own html/.htaccess file (it's a symlink to myapp/public/.htaccess, remember?) and replaced "dispatch.cgi" with "dispatch.fcgi" on the rewrite rules described in step 7 above. Finally, I updated the file "dispatch.fcgi" to include the same "use lib" I did for dispatch.cgi, deleted dispatch.cgi and voilá :)

Done!


Again, Media Temple recommends that any serious app go to their dedicated virtual server (dv) solution. Still, the current gs setup should be enough for you to have some fun and play around with possibilities before moving to a more expensive plan.

There are tons of other cool things you can do, such as creating remote git repositories and making hooks to do automatic deploys, but that's a bit out of the scope of this tutorial :)

Have fun, and happy Perl hacking!

Monday, 8 July 2013

Scala for Perl 5 Programmers



Maybe you want to contribute to the Moe project. Maybe you want to improve your Perl programming skills by stepping out of your comfort zone - after all, learning new languages is generally considered A Good Thing™. Maybe you're doing it just for kicks, or as a second language, or... well, it doesn't really matter.

When I decided to learn a bit of Scala earlier this year, I was devastated to find that most tutorials are for people coming from Java. Well, I'm not really a Java person, so I decided to write a Scala tutorial for people with mostly a Perl 5 background. As I was learning (and writing), I realized Scala feels very Perlish, with anonymous methods, more than one way to do things, weird operators and even a context variable "_" (looks familiar?)

If you're interested, take a look and let me know what you think. It's all on Github, written in their flavored Markdown syntax. And if you find any mistakes - hey, I'm learning too! - or if you have interesting additions to the tutorial, just send me a pull request :)

Happy hacking!

Saturday, 29 June 2013

YAPC::NA 2013 ROCKED!

[Sorry for the delayed post. I wanted to release this right after the YAPC::NA but just a few days after I got back to my country we got kinda in the middle of something.]

In the live music capital of the world, over 400 Perl hackers from 18 countries got together a few weeks ago for Yet Another Perl ConcertConference. If you're not familiar with Perl and its amazing culture, you might think I'm exaggerating as I struggle to find the proper adjectives to describe just how remarkably incredible it was. So, instead, I'm going to describe a few of the things that happened there and how they unfolded from my own perspective. Since everyone's experience is a bit different, I look forward to seeing your post too!

tl;dr - If you've never been to a YAPC before, I strongly recommend you do so. You'll never forget it.

Nothing Like a Little Rock 'n' Roll Before The Conference Starts

I really enjoy Rock/Jazz/Blues, so Austin is one of the great places for me to be. I was a bit sad though knowing that Monte Montgomery, one of my all-time favorite guitarists, and one pretty active in the Austin scene, was not giving a concert while I'd be in town. I even bitched about it on Twitter, like you're supposed to do. And guess what? Just a couple of days before my trip, and two weeks after my rant, Nutty Brown Cafe replied to my tweet and made my trip even more worthwhile!


I was really excited, and it was even more than I hoped for! Nutty Brown is a great place, with great beer and an awesome open air stage in the back. And Monte is even better live! Seriously, I tried to find the person behind the @nuttybrowncafe to thank her (they told me it's a "she") personally, but she wasn't there at the time. I dreamed of seeing this dude perform live for years, and if not for that single tweet I might never have. I guess this whole social network thing works, eh?

But I digress. Back to the conference!

Arrival Dinner

The arrival dinner was at a place called the Hula Hut, serving us some great drinks and TexMex food "with a surfer twist". I'm not really sure what that means but food was delicious, and I think everybody had a great time. I know I did!


Hook 'em Horns!

The venue was in the University of Texas, home of the Texas Longhorns, a very engaging athletics program for students. Everywhere you looked you'd see someone wearing one of their orange t-shirts. It always amazed me how the USA is so into sports (not just american football, but every single sport they can find), and now I got to see that passion first hand. It might be a no-brainer for you, but as a brazilian, I know my country mostly just cares about football, and even so we don't have our college competitions shown on prime time (or any time, for that matter).


Also, as a metal fan, the "Hook 'em Horns" hand gesture was quite familiar (and a bit funny to see in statues and such).

Christmas in June

One of the reasons I look forward to the YAPC every year is because I get to see a lot of the good friends I made over the years in the Perl community, friends I share a lot in common with but whom I get to see only once or twice a year, if I'm lucky.  Being able to hang out with people like sawyer, auggy, ribasushi, perigrin, sartak, ingy, liz, wendy, apeiron, abigail, frew, dha, stevan, genehack, rjbs, jayhannah, hobbs, karen pauley, hugmeir, mdk, mst, nperez, drolsky, jim keenan, nick patch, paul mantz, scrottie and so many others... just being next to these great, fun and sparkling minds already makes this giant trip worthwhile.

I also had the opportunity to meet other great people, some of which I only knew online. Tobias Leich (FROGGS) has been a partner in crime in the Perl SDL project for several years now, but this was the first time we met in person. He showed me a few pictures of his kid and we talked about some of his cool new Perl 6 hacking. He's not only a great guy but an excellent hacker, even more awesome in person than I expected - and I have pretty high standards! =P

I also spotted the great Tokuhiro Matsuno next to (the also great) Xaicron on the first day and introduced myself, since I'm a big fan of his stuff. I actually mixed them together at first (ごめんね!!) but I blame it all on Xaicron's remarkable orange glasses, which I remembered from his gravatar. I also let Matsuno-san know about a recent issue some people were having when trying to report test results for his Minilla app to CPAN Testers through cpanm-reporter, and he fixed it on the same day! Wow! A few days later at the speaker's dinner I'd be laughing with them (and Karen Pauley, and Shawn) about some very weird (miss)translations we have between languages such as portuguese and japanese. I only talked to them briefly but I hope they've has as much of a good time as I had, and hope to see them again (and even more Japanese developers) on future editions of YAPC::NA.

Other great people I met there for the first time whom I'd only talked to online included Naveed "Ironcamel" Massjouni and Al Newkirk. Al was in fact one of the first people I interacted with over on irc.perl.org, so I was really glad to finally meet him. Oh! And at last I got to met Karen Etheridge, who's been stalking me online for several months now. Fine, fine, I was the one stalking her - it was driving me mad that all of a sudden she was the release manager of a ton of modules I loved and used on a daily basis and yet I had no idea who she was! Turns out she's not only a great developer but also a very nice person, so yay!

I really can't explain it. There's something about the Perl community that feels like family - for better or worse. So, to me, attending the YAPC is like walking into one big Christmas party. I just love it! :-)

As any family this big, I missed a lot of people who couldn't be there this year, and hope to see them in other conferences or back at the YAPC::NA next year!

CGI.pm is Dead. Long live CGI.pm!

One of the (many) interesting highlights of this year's conference was the whole discussion of CGI.pm - specifically whether it should be removed from the Perl 5 core or not. SawyerX did a lightning talk making a strong case for "killing" CGI.pm. Perrin Harkins, on the other hand, came up on stage with a lovely tribute to all the great stuff that CGI.pm acomplished back in the day and bid it a fond farewell from the core. Later on the conference, Casey West went totally the other way, not accepting CGI.pm's fate/doom in a very funny rendition of his arguments which included sarcasm, the shortest wiki in the world and doing a handwalk on stage wearing nothing but a swimsuit!

The Keynotes

Mark "mdk" Keating gave the first keynote of the conference which secured his Diva status in the Perl community with a brilliant, brilliant talk called Perl of Christmas Past, in which he dwells on the history of the language and other fun facts. For example, did you know that Perl 1.0 was released on the very same day as Final Fantasy 1?! Did you know Perl has been used in Mars? I don't think I'll ever forgive myself for not making it in time to watch it live, but thankfully the conference recorded it so I was able to watch it afterwards, and you should too!

In the evening, Larry Wall gave an entertaining and enlightening (as usual) keynote. It was also the first time (I think) where he openly talked about his recently found cancer and discussed his legacy and Perl's future as a language and community. Not a lot of people know this, but back in 2009 I lost my father to cancer, so I really related to a lot of the things he said. The talk was extremely passionate and got him a well-deserved standing ovation. Incidentally, Perl is being used throughout the world in cancer research, helping to achieve some great breakthroughs in the fight against it over the past several years now. I truly hope Larry wins this battle, and I'm pretty sure the entire community is supportive and wishing him and his family all the best.

Tuesday, Stevan Little gave the "Perl: The Detroit of Scripting Languages" keynote, in which he evolved his OPW talk with new experiences and insights on the present of Perl, stuff we love, stuff we hate, and where to go from there.

Finally, in wednesday Matt S. Trout gave his "Velociraptor of Christmas Future" keynote, also filled with nice insights and perspectives on what the future holds for Perl as a language and as a community, all in mst's unique shoutingtalking style ;-)

cPanel Party Night!

pics, or it didn't happen ;)
On the very first day of talks, cPanel greeted all attendants with an open bar(!!) party at the Recess Arcade Bar down on 6th street, featuring a very nice band called The Spazmatics playing some cool covers. I had a great time there and even got to finish TMNT with 4 other guys (hey, it's an Arcade Bar after all!). It really got me back to my childhood. Thanks, cPanel! You guys are awesome!

Later that evening some of us moved to the pub in front of Recess to chat for a bit - the arcade bar was great, but also pretty loud. This is where I was exposed to the InfoWars magazine for the very first time. I... I... let's move on :-)

Game Night

Tuesday's talk ended with a great dinner with ribs and tex-mex at a place called Salt Lick BBQ - yet another fully paid for dinner, courtesy of our amazing sponsors! The conference organizers even got us private buses(!) to drive everybody there. Food was great, with tasty ribs, lots of side dishes and even dessert. In the meantime they also featured a caricature artist drawing everybody brave enough to ask for it.

During dinner I engaged in a great discussion with Shawn O'Connor, CTO of Perceptyx, about companies and employee engagement. As a manager myself, it was very reassuring to talk to another peer about the rights and wrongs of steering a company and motivating teams. We also had a great talk about football because, well, Brazil :-)

Then it was time for Game Night! Some people went outside to enjoy the atmosphere and play Jenga, and wound up creating pretty big towers - and even more spectacular crashes! Others stayed inside for some board and card games, or just to chat. At one point I was even hit in the back by a frisbee, so I guess there was that too :)

I for one enjoyed a very nice D&D(ish) game by Rik set in the future, who in the end revealed it to be a Star Trek Next Generation spin-off adventure. I'm not a hardcore trekkie but it surprised me that none of us could actually spot his (now obvious) hints and easter eggs throughout the adventure.

Talks, talks everywhere!

This year I spend most of my time in the "hallway tracks", bouncing from room to room and engaging in great conversations with fellow developers. I also volunteered to help the organizers (brown shirts FTW!) so I hung around the main LBJ desk helping Chris "perigrin" Prather as best as I could.

That doesn't mean I didn't enjoy a few of the 80+ talks (not to mention lightning talks) spread through the 3 days of the conference. I liked them all very much, but I feel I need to mention at least two of them here: first, Ricardo "rjbs" Signes' talk on the future of Perl 5 ("Postcards from the Edge") was great, debunking a few myths and unveiling a lot of how things are thought through in the core development of P5P and what we can expect from Perl 5 versions in the near future - including long-awaited signatures! Second, Augustina "auggy" Ragwitz' talk on extremely easy ways to contribute to Perl 5 (and dip your toes in Perl 5 development in general) was very well thought of and presented, and I really think it can break some entry barriers and reach developers having their first contact with Perl.

The Lightning Talks sessions were also very entertaining and fun. It amazes me how people in the US just rush to line up and talk about all the cool stuff they're doing with Perl. In Brazil, at least, most YAPC attendants don't seem attracted to giving lightning talks. Maybe they're too embarrassed, I don't know. Still need to think about ways to change this around here, I just love lightning talks too much :-)

Worldwide Job Fair

One of the high points of the conference is the job fair, specially if you're looking for new challenges, moving opportunities or simply a larger paycheck. Perl-centric companies from all over the world come to YAPC::NA to promote their business to potential employees and recruit them.

I'd like to take this moment to thank cPanel, Whitehat Security, LiquidWeb, Linode, AthenaHealth, The Game Crafter, MediaMath, Shutterstock, Bluehost.com, Booking.com, HostGator, NextGen, GlobalNOC and everybody else at the job fair (really sorry if I missed your company, just let me know and I'll update this) for being there, not just for the conference, but for Perl developers all over.

Seriously. These companies built their businesses around Perl and heavily rely on it for striving in the competitive market every single day. You should definitely support them if you can. And if you're looking for a full-time Perl job, please send them your resume.

"You're not my community. You're my family."

In between the Lightning Talks, while speakers were setting things up, people had the opportunity to make quick (~30s) announcements on whatever it is they felt like it. On the last day, Ribasushi poured his heart out thanking the Perl community for sponsoring his trip to the event via Crowdtilt. It was very touching, specially since I kinda feel the same way.

My Lightning Talk

For the first time ever, I went to a Perl conference committed to not giving any talks. I failed :) Jim Keenan approached Brian "Hugmeir" Fraser and myself during the pre-conference dinner asking us to follow his fiendish plot of having a Spanish & Portuguese lightning talk at the conference. We were really excited of being able to share a bit of our language and culture with everyone (Hugmeir is from Argentina), specially in a US state so close to Mexico. We were also terrified, thinking there was no way we could pull this off, and that people would just have to settle with a 5 minute "wtf" talk in not one but two different languages. So I figured what the hell, let's make it a comedy act and have the slides in english pretending they're part of the english-only audience and not getting anything we say. Hugmeir was totally up for it and we even got guest stars Genehack and Sawyer to help us flip the slides. As we started talking the crowd quickly turned its initial confusion to giant bursts of laughter. We had a great time and the response has been overwhelming!


We even got featured on the Shutterstock YAPC::NA blog post! This was also pretty big for Hugmeir as it was his very first talk. Thanks everyone!

Bingo \o/


The conference itself is filled with small pleasures, and one of them is the YAPC Bingo. Every attendant gets a card filled with stuff to do during the conference. Every time you do one of them, you cross it on the card. This year the only one I missed was the Bad Movie BOF, which I heard was great and fun as usual. This was a very tough choice for me as I really enjoy watching crappy movies. Besides, David Adler is an amazing bad movie connoisseur and a fun guy to hang out with in general, but when so many things are going on, you eventually get caught up in them. Also, since I don't see many of these people throughout the year, I just couldn't bear to let go of the hanging out and chatting.

Also, Sawyer managed to reach out to first-time attendants, finding tons of seasoned volunteers to help them with whatever Perl help they needed for their projects, provided they showed their YAPC Bingo cards with at least one full row/column/diagonal line completed.


That's it! I'm pretty sure I missed a lot of highlights, so make sure to blog about them if you can, or just let people know the stuff you liked in the comments below. A huge THANK YOU is in order to master chief Todd Rinaldo and the great Austin Perl Mongers for organizing such an incredible conference.
 
And now I can't wait for YAPC::EU, YAPC::Asia, YAPC::Brazil..... see you out there!

Monday, 14 January 2013

Perl in 2012

I know, I know, I'm two weeks late. Sorry! I scheduled this to go live on January 1st but obviously something went wrong =X

Happy New Year, Perl folks!


2013 is already here, and if it's even half as good as 2012, we're in for quite a treat. So, without further ado, here are the highlights of the Perl world in 2012!

Perl Turns 25!


One score and five years ago Larry Wall brought forth on this world a new programming language, conceived in liberty and dedicated to the proposition that there is more than one way to do it.

To celebrate this amazing ride, Mark Keating wrote an incredible retrospective of the Perl community and its history. It's long - well, it's been 25 years! - but it's definitely worth checking out. This is, in fact, one of the reasons I like Perl so much: it's not just a (pretty damn good) tool to get the job done. It's 25 years of culture; of progress, adaptation, community, friendship, of good days and bad days, exhilaration, frustration, love, hate; of being a part of something that has a life of its own, something that's not blunt or dull, and that attracts brilliant people willing to share the knowledge and make the most amazing things.


The Map of CPAN



Speaking of "living and breathing", have you seen this movie by Grant McLean?!



It comes from mapofcpan.org, a very cool website made to let you explore all the Perl modules uploaded to CPAN. It also offers some nifty sightseeing tours where you'll see recent uploads, profile updates, leaderboards and loads more, all thanks to the awesome and ever-increasing MetaCPAN API.


A Brand New Perl Data Language


Just because most of the world is now focused on web technologies, it doesn't mean that several other niches have ceased to exist. Quite the contrary, actually. 2012 saw a lot of new developments in science, with NASA's Curiosity rover on Mars, Voyager 1 leaving the Solar System, and CERN observing the Higgs boson.

Did you know NASA uses Perl as one of its languages? Some of it is even open source! CERN and many other labs also rely on Perl for processing information and making separate systems talk to each other. But I digress. My point is, if you're doing science with Perl you're probably doing some heavy math. And if you're doing heavy math with Perl you're probably using - or at least have heard of - the Perl Data Language suite.

For those of you that don't know, PDL gives standard perl the ability to compactly store and speedily manipulate the large N-dimensional data sets that are the bread and butter of scientific computing. In 2012, we saw the release of PDL 2.4.10, a long-awaited version that includes support for automatic multi-thread parallelisation, data structures of over 2GB and POSIX threads. If you want to check it out, they also released the first draft of the PDL Book in pdf, which does a great job at complementing the already thorough PDL documentation.


New Milestones for Perl Quality Assurance


This was a great year for the CPAN Testers. Not only the QA Hackathon yielded some awesome results, but the amount of received tests reached the 20 million mark! In fact, the year closed with over 27 million reports. Keep them coming!

Another important number was reached by the CPAN::Changes Kwalitee Service, created by Brian Cassidy to promote a standard format for Perl module's changelog to enable automated parsing and analysis. In September, the service reached 10,000 readable Changes files on CPAN. That's almost 40% of CPAN! It's still a long way to go but this steady increase in compliance goes to show how important the initiative is.


Startups and Perl



We all know the story of how Yahoo! was created with Perl, and how high-traffic websites such as Amazon, Craiglist, IMDb and even the BBC use Perl extensively as part of their core business in mission critical applications and tasks. But it seems these days people are focused on web startups and how to be the next Silicon Valley sweetheart, so it's worth checking out whether modern Perl is still up for the task in a world with so much competition and so many different dynamic languages to chose from in your business.

I could mention the tons of modules on CPAN that do all the heavy lifting for you and let you worry just about the stuff that really matters to your business, delivering your products on time and under budget. I could mention all the conferences and support - free or paid - that you can get (in fact I am, just not now). I could even mention how businesses like Booking.com built their multi-billion empires taking advantage exactly of all these features, using Perl as their main language. But this is a 2012 roundup so, instead, I'm going to share some 2012 enterprise Perl news:

Do you know Moonfruit, the website and shop builder created by couple Joe and Wendy White? They've been around for some time, but only lately they've made a real push in innovation, creating a brand new site and interface using the Perl Dancer framework, and focusing on some dazzling pre-built designs for their users to chose from. The reward? In May 2012, Moonfruit was acquired by Yell for $29 million USD!



On the other side of the Atlantic, the startup search engine DuckDuckGo hit yet another milestone in 2012, with over one million direct search queries per day. In a web ruled by search monsters such as Google, Bing, Yahoo!, Ask and AOL, that's pretty impressive, and represents a growth of over 500% for them!

Speaking of impressive, the Perl-based global image marketplace Shutterstock surpassed 20 million images in its collection and completed its initial public offering on the New York Stock Exchange in October. Shutterstock's NYC headquarters revealed the financial results of the third quarter, with a revenue of $42.3 million USD, a 36% increase from last year! And you can see why Shutterstock chose Perl directly from the horse's mouth =)

Finally, you might have heard of JT Smith's startup, The Game Crafter. Since 2009, this Perl business created an industry as it became the world’s first web-to-print game publishing company, offering a print on demand game publishing service. In 2012, the company grew by 318%, selling nearly 20,000 board games and almost 400,000 loose game pieces!

So, what are you waiting for? Who knows, maybe next year I'll be talking about YOUR Perl-based startup company ;-)


Conferences & Workshops


As previous years, 2012 was filled with Perl events!

Starting off the season in January was the Orlando Perl Workshop (OPW), also known as The Perl Oasis. This conference holds a very special place in my heart for being my first ever international Perl event. I definitely recommend you check it out!

February saw another edition of the Israel Perl Workshop, with some very interesting talks. Shlomi Fish was kind enough to write a very nice review of the event for those of us that couldn't attend.

In March we had the ever-great German Perl Workshop, while Paris hosted the Perl Quality Assurance Hackathon. I had the privilege of participating in the QA Hackathon, helping out in what I think are some of the most important bits of the Perl ecosystem. Hopefully this year I'll be able to attend as well, finish what I started and try and help as much as I can.

April was the DC-Baltimore Perl Workshop, where you could chose between great talks, beginner training courses or a day-long Perl hackathon. Almost at the same time, in Europe, the Dutch Perl Workshop also held a very high quality, full day event.

May held a joint Perl Mova (Ukranian Perl Workshop) and YAPC::Russia in Kiev. This was special one for them because they had over 200 people registered, several guest speakers and attendees from over 13 countries. Even more so, Kiev was chosen to host YAPC::Europe in 2013! Congratulations!

In June the worldwide Perl community stood in awe as J.T. Smith presented us with one of the greatest YAPC::NA of all time! Beautifully organized by himself and the Mad Mongers, the conference served over 400 attendees from the entire World with an incredible infrastructure, a full week of modern Perl talks, training courses and hackathons (including a ginormous hardware hackathon courtesy of Robert Blackwell), plus the job fair, game night, bad movie night... Wow. Just... wow.

After the YAPC, many Perl developers traveled to DuckDuckGo's HQ in Philadelphia for the 2012 Quack and Hack event, where the company held Perl talks and hackathons for their public API.

Later that month we also had the French Perl Workshop, a 2-day conference that's attracting more and more people each year, proving once again that the French Perl community is one of the most actives in Europe. Allons-y!

As usual, July is OSCON time, and once again we had lots of great talks on the Perl track.

In August we had the always impressive, week-long YAPC::Europe, this time in Germany. Sadly I couldn't make it but I heard it was *HOT*, in both senses of the word ;-) Just before that, the brilliant duo Liz & Wendy organized the Perl Reunification Summit, in the city of Perl, where a lot of very important issues were discussed regarding the future of the Perl community as a whole. Finally, in late August, the Oslo Perl Mongers invited everyone to their Moving to Moose Hackathon, in Norway, where attendees got 4 full days of hacking, discussions and fun!

it's over eight hundred!!!!!!

September was the month of YAPC::Asia, the biggest Perl conference in the world, and in 2012 they broke their own record: over 800 people attending!

Once again, the incredible Daisuke Maki, his partner in crime Kushii-san and the entire staff of 43(!) volunteers pulled off an amazing conference. Maki-san, as usual, posted a very nice roundup of YAPC::Asia 2012 in his blog, so make sure to check it out!


Still in September, the traditional Portuguese Perl Workshop was held in Braga and included a Moose tutorial and several very interesting talks.

October was a month filled with Perl. We had the Italian Perl Workshop, the Nordic Perl Workshop, the Korean Perl Workshop, and YAPC::Brasil. So many conferences, so little time! I, of course, was in São Paulo for the YAPC::Brasil, and not only saw some great talks and participated on the OpenData hackathon, I also saw history being made: Florian Ragwitz released Perl 5.17.5 live on stage! This was not only the first ever live Perl release, but also the first release in South America! Many thanks to Florian and the entire Perl Core Team for letting this happen \o/

In November we had the Austrian Perl Workshop and the always amazing London Perl Workshop, UK's free and premier Perl event, holding over 200 people each year!

Finally, closing the calendar, DuckDuckGo's Quack and Hack Europe introduced several people to some modern Perl basics in beautiful Paris, while on December 22nd St. Petersburg held the Saint Perl conference with over 40 people.


In the Year of the Olympics, a new World Record



Back in January 2012 I made the terrible mistake of letting rafl know about my evil plan of getting the World Record for most YAPCs attended on a single year. I wanted to cover all the 5 current YAPCs - NA, Russia, EU, Asia and Brasil - but I had just switched jobs and was unable to do it this time.

He did.

rafl> really, there's no way I'm letting you win this whole most YAPCs a year thing ;)

So there you have it. Florian Ragwitz holds the World Record in most YAPCs attended in a single year. I love this crazy bastard, and I can attest that, as legend is told, he is pretty much everywhere :)

(ssshh! By the way, if you want to steal his record (I'm sure gonna try!), I heard there's a new YAPC coming up from down under!)


White Camel Awards


The White Camel Awards of 2012 went to Renée Bäcker, Jim Keenan and... myself! Wow, I don't even know what to say except a very big "Thank You" to everyone. Not just a huge honor to me, I feel this award was particularly important as it was the very first White Camel given to someone from Latin America, and hopefully it will help reduce even more the barriers between Perl communities worldwide, showing people that there are some nice things going on on this side of the equator too :D

I would also like to take this opportunity to congratulate both Jim and Renée for the astounding work they've been doing over the years. Very well deserved recognition, guys!


Free/Low Cost Perl Training


This year Dave Cross started his Perl School project, bringing low cost and high quality public Perl training into the UK. In 2012 he managed to give three training sessions, two "Modern Perl for Non-Perl Programmers" and one "Object Oriented Programming With Perl and Moose". Cool!

The Miyagi University in Japan also started a series of free special courses that featured several outstanding Perl Mongers like Goto Eikichi (egopro), Daisuke Murase (typester), Yusuke Wada (yusukebe) and Dan Kogai (dankogai). I think they ran into some trouble due to earthquakes (my japanese is not very good), but it was still a great initiative and I'm eager to know how that went.

And if you're into online training, you should definitely check out vti's Perl Tuts, a website where you can learn modern Perl 5 through several tutorials and, best of all, try your code online!

brian d foy also started a series of Learning Perl Challenges, a very nice resource for beginners wanting to test (and improve) their skills.

Finally, Gabor Szabo started the Perl 5 Maven website, in which he pushes several articles and tutorials of different shapes and sizes, aimed to improve your Perl expertise. Also, in April, his Perl Weekly newsletter hit 3,000 subscribers, wow!



2012 Stats


Perl 5 tickets opened: 853
Perl 5 tickets resolved: 1036

Perl 6 tickets opened: 433
Perl 6 tickets resolved: 473

Sweet! We've finished the year with a very nice positive margin :)

Between stable and development, Perl 5 saw 21(!) new releases, including the new 5.16, with Unicode 6.1 support, several performance enhancements and much more.

Meanwhile, the Rakudo Perl 6 team kept the promise of 1 new "Rakudo Star" release each month (except for the March release but they had a good reason), and now the project has an MSI Installer for Windows! You can download the December release here (or here, for the MSI). As you can see on their feature comparison page, almost everything in the spec is already working. Congratulations!

The Perl Foundation also released their 2012 Year End Report, and what a great year it was!

Speaking of stats, you should also check the summaries for the 20 best Perl questions of 2012 at Stack Overflow, and the 2012 most voted distributions on MetaCPAN, both courtesy of Miguel Prz. Note that Miguel has been doing weekly summary posts about this, so if you want to keep track of that make sure to subscribe to his feed!


New Perl Monger Groups

Cluj Perl Mongers' cool "dracula" logo

In 2012 the worldwide Perl community welcomed 8 new Perl Monger groups: Mumbai, Weston, Cluj, Niigata, Baku, Shijiazhuang, White Plains and Duesseldorf, not to mention Robert Blackwell's Hardware Perl Mongers, for all of us who enjoy using Perl to play with Arduino, Rasberry Pi and other devices.

We also saw a huge facelift in the Perl Mongers website. Much better now! Code's on github if you want to contribute. I heard patches are very welcome :)

Finally, still in the topic, Vyacheslav Matyukhin made a Perl Community Motivation survey, and a lot of people submitted their input as to what drives them. Results are in, and worth a peek.


Got Book?



Back in February we saw the Camel Book, "Programming Perl", go into its 4th edition. This is a much-anticipated update to what's considered by many as the bible for Perl 5 and a must read for everyone, and I'm really glad to see it fully cover 5.14!

Later in 2012 the very cool "Intermediate Perl" by Randal Schwartz, brian d foy and Tom Phoenix also got an update with its second edition! I had the privilege and the honor of being part of the reviewing team for this one, and I was very impressed at how much got updated. If you liked "Learning Perl", you're gonna love "Intermediate Perl".

Curtis "Ovid" Poe made his book author debut this year, with his "Beginning Perl", a modern introduction to Perl programming that even includes material for instructors. You can browse some of its contents here but, if you like it, I highly recommend you get a copy.

Speaking of modern, chromatic's "Modern Perl" book was updated in January and is available in several different formats, including some free ones (like EPUB)!

Last but not least, Miyagawa's Plack Handbook was released in Japanese and English, containing 24 useful short articles that explain what PSGI is all about and how to adapt Plack to existing web applications. And guess what? For a few days it was ranked #4 on Kindle Japan on the Computer/IT category. Omedetou!


Interesting New Dists



This is in no way comprehensive, but I also wanted to share some interesting new distributions that reached CPAN in 2012. Check them out!

  • archlib, by Chad Granum is a proof-of-concept that lets you add tar archives to @INC.
  • Damian Conway's new Regex::Debugger makes debugging regular expressions much easier and even includes a command-line REPL called rxrx to visually and interactively check your regexes.
  • Tokuhiro Matsuno's Test::Pretty (ab)uses colors and unicode characters to make your test output beautiful and much more readable!
  • sqitch, by David Wheeler, is a self-entitled sane, standalone database change management application written in Perl that supports native scripting, dependency resolution and iterative development.
  • Ever wanted a modern web-based Perl editor? Try Farabi, an experimental "fork" of Padre made by Ahmad Zawawi and making heavy use of JavaScript.
  • Although not a new dist but also very noteworthy, Plack, the module that revolutionized Perl web frameworks, has hit 1.0! After seeing so many companies adopting it for production, Miyagawa-san decided it was time to formalize it as stable. He also set a core team to make it even easier to support and further develop. This is very good news, and you can check out the full announcement here.

How about you? Discovered some cool module this year?


Donations


Early in 2012, Craiglist donated $100,000.00 to the Perl Foundation. According to their CEO and former Perl Hacker Jim Buckmaster, "craigslist has gloried in and relied upon Perl for most of its software development for more than a decade. craigslist Charitable Fund is honored to recognize the wizardry and generosity of the Perl community, help ensure the ongoing maintenance of Perl5, and contribute to the future evolution of Perl."

Many many many *many* thanks to Craiglist, and to all the companies and individuals that continuously help the language and its community. 

And if you want to donate as well, both The Perl Foundation and the Enlightened Perl Organization will definitely appreciate it, and even let you chose specific destinations to your money (like a particular conference or to the CPAN Testers). Every penny counts!


Wrapping Up



Well, that's it for 2012. Sorry for the long blog post but, as you can see, it's been a really great year for Perl ;-)

Let me know if I missed anything, ok? And keep on making great things with modern Perl!

Wednesday, 5 December 2012

Are you relying on hash keys being ordered?

tl;dr - if you rely on Perl 5, and plan to eventually upgrade to the upcoming 5.18, do this *now*:

  1. Install 5.17.6 (you do use perlbrew, right?);
  2. Try your modules and apps in it (you do have tests, right?);
  3. If  anything breaks, it's likely because you're relying on keys(), values() or each() being in some particular order. You really shouldn't, so go sort() your keys or something :)
  4.  If some CPAN module you depend on suddenly fails on 5.17.6, make sure to let the author know;
  5.  Spread the word!

On Hashes & Security


The Perl 5 core team has always put security as one of its top priorities. To put things under perspective, in late 2011, an algorithmic complexity remote denial of service attack (original paper, advisory, slides, video) was found on major language implementations like PHP, Ruby, Python, Java, even JavaScript. It's been fixed in Perl 5 since... 2003.

That was then. What about now?


Still thinking about security, Yves Orton pushed some important changes these past few weeks, changes that are going into perl 5.18.0. Among other things, to quote 7dc8663964, it:

  • Introduces multiple new hash functions to choose from at build time. This includes Murmur-32, SDBM, DJB2, SipHash, SuperFast, and an improved version of the original One-at-a-time.
  • Rips out the old HvREHASH mechanism and replaces it with a per-process random hash seed.

Optimizations aside, the ability to change hash functions easily is important because, if, for whatever reason, the active function is found vulnerable to an attack, you don't have to wait until the Perl Core Team (or your specific vendor/system) releases a fix: just recompile your perl setting another hash function as default.

The important bit, however, is the per-process random hash seed. Until now, perl was using a not-so-great hash seed, one that was set during compilation. All hashes would use this seed, and if a collision attack was detected it would trigger a rehash, where every item in the hash would have its hash value recalculated, with corresponding effects performance and memory. Of course, when too many collisions were found, the rehash would switch to a random seed instead.

Now, after this change, every process is guaranteed to use a random seed.

Hash randomization should make perl even more robust to complexity attacks, and with simpler code. But, as you may have predicted, there's a side effect to it: the order of hash keys changes more often than before.

Sweet! But, what does it mean to my code?


As it is stated in perlsec since version 5.8.1 (that one from 2003), Perl has never guaranteed any ordering of the hash keys, and in fact the ordering has already changed several times throughout its history. The problem, however, is that a lot of developers end up inadvertently relying on hashes being ordered, or rather in some random but constant order, simply because that particular order worked on their machine. Talk about a subtle bug!

This may not be your case, but you should check nonetheless. Andreas König, Father Chrysostomos and the rest of the P5P/CPANTesters gang have gone through the enormous effort of testing several major CPAN distributions for this and letting authors know whenever it failed a test while running on a patched version of perl, but they can only do so much, and there's *your* code to test, too.

You know, code your app runs, code that you haven't checked to CPAN.

Oddly enough, it looks like most of the found issues are on test cases themselves, tests that expect keys() to be in a particular order. Now, keys() is guaranteed only to return items in the same order as values() or each(), and even that is only true for the same process, so make sure you're not shooting yourself on the foot.

LIES! My code is perfect, you're the ones that broke Perl!


Well, not really. Like I said, it's a subtle bug, one that might be hitting your production code right now, but only on some very specific scenarios, and be very hard to reproduce and debug. If you don't trust me, there's a very simple experiment you can run on your system perl:

First, let's create a simple one liner that creates 15 key/value pairs, and print them on the screen:

   > perl -E 'local $,=q[, ]; $hash{$_} = $_ for 1..15; say keys %hash'
     6, 11, 3, 7, 9, 12, 2, 15, 14, 8, 1, 4, 13, 10, 5

You may have gotten a different order (did you?), but you'll probably get that same order no matter how many times you run it:

   > perl -E 'local $,=q[, ]; $hash{$_} = $_ for 1..15; say keys %hash'
     6, 11, 3, 7, 9, 12, 2, 15, 14, 8, 1, 4, 13, 10, 5
   > perl -E 'local $,=q[, ]; $hash{$_} = $_ for 1..15; say keys %hash'
     6, 11, 3, 7, 9, 12, 2, 15, 14, 8, 1, 4, 13, 10, 5
   > perl -E 'local $,=q[, ]; $hash{$_} = $_ for 1..15; say keys %hash'
     6, 11, 3, 7, 9, 12, 2, 15, 14, 8, 1, 4, 13, 10, 5
   > perl -E 'local $,=q[, ]; $hash{$_} = $_ for 1..15; say keys %hash'
     6, 11, 3, 7, 9, 12, 2, 15, 14, 8, 1, 4, 13, 10, 5
   > ...

What happens, however, if your code adds a 16th key and then, realizing its mistake, removes it right afterwards (highlighted code below)? There are still 15 keys, the very same 15 keys as before, so surely they'll be in the same order, right? Right? Wrong:

   > perl -E 'local $,=q[, ]; $hash{$_}=$_ for 1..15;
             $hash{16}=16; delete $hash{16}; say keys %hash'
     11, 7, 2, 1, 13, 6, 3, 9, 12, 14, 15, 8, 4, 10, 5


This can happen anywhere, like when reusing a hash variable:

    sub init { ( 1=>1, 2=>2, 3=>3, 4=>4, 5=>5 ) }

    my %hash = init();
    say "original: " . join ', ' => keys %hash;
    $hash{$_} = $_ for 6..100;

    %hash = init(); # restores original values
    say "original? " . join ', ' => keys %hash;


This is what I get on my good old 5.14.3:

    original: 4, 1, 3, 2, 5
    original? 2, 1, 3, 4, 5


As you can see, it's a real problem and it could be lurking in your code right now. What Yves' patch does is simply expose the issue more explicitly to you. This is a good thing, because, aside from the extra security protection, it will let you spot buggy code much easier. If you try that previous one-liner on 5.17.6, you'll get a different key order every time you run it:

   > perl -E 'local $,=q[, ]; $hash{$_} = $_ for 1..15; say keys %hash'
     1, 5, 15, 12, 6, 4, 10, 9, 3, 13, 7, 14, 11, 2, 8
   > perl -E 'local $,=q[, ]; $hash{$_} = $_ for 1..15; say keys %hash'
     5, 11, 7, 3, 15, 6, 12, 2, 13, 9, 8, 14, 10, 1, 4
   > perl -E 'local $,=q[, ]; $hash{$_} = $_ for 1..15; say keys %hash'
     2, 15, 14, 13, 5, 1, 9, 10, 3, 11, 6, 8, 12, 4, 7
   > perl -E 'local $,=q[, ]; $hash{$_} = $_ for 1..15; say keys %hash'
     8, 2, 14, 10, 1, 9, 4, 3, 6, 15, 5, 13, 7, 12, 11


Uh-oh... looks like my code is broken.


Not to worry, the fix is usually pretty easy! Look for the failing test and see if whatever is being tested calls keys(), values() or each() at some point. You'll likely want to sort() the results or change your code algorithm to something more deterministic.

I don't really have that many tests... What can I do?


Look for calls to keys(), values() or each() in your code, and make sure they are not relying on the order of the elements being returned. It is ok to do something like:

  my @keys   = keys %hash;
  my @values = values %hash;
  say "hash key $keys[3] is $values[3]";

because, as I said before, keys() and values() will always use the same order for the same process, whatever that order is. However, this is not ok:

  if ($keys[0] eq 'some_key') {
     ...
  }

simply because there's no way to guarantee the order of the list returned by keys(). The code above might have worked, however, if you always sorted the returned value, like so:

  my @keys = sort keys %hash;
 

Indirect usage


Sadly, your code is not safe just because you don't use those functions (or have them properly sorted). Sometimes you expect lists of values from external modules, and those lists might be affected by the change. So make sure you look for arrays that are populated by external functions and see if you rely on their order being a particular one. For example, you might have code like:

   my ($name, $age, $rate) = Some::Module->new->get_list( 'some_user' );
   
And, within Some::Module, you'll find the suspect:

  sub get_list {
    my ($self, $username) = @_;
    return values $self->{data}{$username};
  }

Make a failing test for it, push a fix, rinse and repeat :)

I hate this! Switch it back!


This is hardly going to happen. Remember: hash randomization is a good thing! Please take another look at the sections above and try to fix your code. If you need help, ask for it at the usual places, like your local mailing list or IRC - heck, even Facebook has a Perl group!

But if you really really really need the previous behavior, you can simply stick to 5.16, or try compiling perl defining PERL_HASH_FUNC_ONE_AT_A_TIME_OLD to simulate the old algorithm, but the entire rehashing mechanism is gone, so specifying your own PERL_HASH_SEED value is probably as close as you'll get :)

Many thanks to the nice folks at P5P for their continuous effort in keeping us safe!

Wednesday, 30 May 2012

Moving modules across perlbrew installations

This short post was triggered by a conversation I had on Twitter with a friend:


He does strike a nerve there. Ever since Gugod's amazing perlbrew came to life, installing your custom perl - or several perls for that matter - is *really* easy. However, a new problem surfaced: updating your installation so your applications work again. This is, of course, per design. After all, perlbrew is supposed to give you completely separate installations, and this includes installed modules.

So, what can you do? Copying your lib directory is out of the question if you have any module that does XS, and chances are you probably do. I'm also not sure if an external local::lib directory would play nice either.

Luckily, there's a quick-and-dirty recipe to migrate your installed modules from one perlbrew installation to another. Say I just upgraded from 5.14.2 to 5.16.0 and want to install the same modules I had before:

    $ perlbrew switch 5.14.2
    $ perl -MExtUtils::Installed -E 'say for ExtUtils::Installed->new->modules' > /tmp/installed.list
    $ perlbrew switch perl-5.16.0
    $ perlbrew install-cpanm
    $ cat /tmp/installed.list | cpanm --interactive

Done!

What I did there was use the core module ExtUtils::Installed to create a list of installed modules in the file "/tmp/installed.list", then feed that list to cpanm. I used "--interactive" because some modules I have (like SDL) ask a few questions during installation, but whatever rocks your boat.

Also worth noticing that depending on the amount of modules you have installed, that last step can take quite a while, so go watch a movie, read a book or something :)

I think this tip is particularly pertinent if you're trying out a Release Candidate (RC) for a future perl release and want to make sure your toolchain builds properly - which is always a good idea.

I'm going to bug gugod for a bit to see if we can come up with something bundled into perlbrew to make this even easier, but for now you can use this :)

Hope it helps! Happy Perl Brewing!

Monday, 9 April 2012

Perl QA Hackathon

Last weekend I was in Paris for the Perl QA Hackathon, a free of charge coding workshop for people involved in Quality Assurance, testing, packaging, CPAN, and other related projects.

I had the best time, met a lot of old friends and made several new ones as well. And we got so much done! It's amazing the amount of work you can do just by sitting next to fellow hackers with the same goals. Coding, debugging, design, feature requests, decision-making, if you had questions or needed help or feedback, all you had to do was look around the room and find the authors/maintainers for a quick tête-à-tête.
Schwern and Apeiron looking at some code
The hackathon was responsible for over 1 thousand man-hours of work on the Perl toolchain ecosystem in just 3 days, and none of that would be possible without all the great companies and organizations supporting the event: The City of Science and Industry, Diabolo.com, Dijkmat, DuckDuckGo, Dyn, Freeside Internet Services, Hedera Technology, Jaguar Network, Mongueurs de Perl, Shadowcat Systems Limited, SPLIO, TECLIB’, Weborama, and $foo Magazine. There were also a lot of amazing people who made donations themselves, like Martin Evans, Mark Keating, Prakash Kailasa, Neil Bowers, 加藤 敦 (Ktat), Karen Pauley, Chad Davis, Franck Cuny, 近藤嘉雪, Tomohiro Hosaka, Syohei Yoshida, 牧 大輔 (lestrrat), and Laurent Boivin. Thank you!

Many thanks are also in order to Laurent Boivin (elbeho), Philippe Bruhat (BooK) and the Mongueurs de Perl, who did an incredible job organizing the event and hosting/feeding us. Merci beaucoup! :-)

You probably already read about some of the great things that happened there. I guess it's my turn to share:

I got there thursday night, after just a couple of minutes lost at Gare du Nord trying to find the right Metro track, nothing my poor french (and a lot of pointing at maps/signs) couldn't solve. I shared the hotel room with Leon Timmermans, who arrived just a few minutes after I had settled in. Turns out Leon is not only a great Perl hacker, but also a very nice guy.

I was a little apprehensive about waking up in time - I want to say "jetlag" here, but truth is I'm just not really a morning person. Fortunately, the excitement of having a joint hacking session with several of my Perl heroes was much greater than my will to stay in bed.

After talking to a few people and finding a nice spot to settle, I was ready to start hacking. I've been meaning to try and add CPAN Testers' support to Miyagawa's great cpanminus for quite a while now, and as it turned out so did he! We talked about some of the details and he was kind enough to help me through part of the cpanminus source code, as we discussed the implementation details.

The first draft of the build.log parser was finished on that same day, so all I had to do was send the email, push things to CPAN and move to my next target, right? Wrong!

As it happens, the CPAN Testers is much more than what is exposed to module authors and users. Its amazing reports and statistics are the result of a delicate and intricate mix of modules, protocols and systems that have to work on all sorts of environment and talk to each other all the time. I have the utmost respect and admiration for people like Barbie, Andreas König, David Golden, BinGOs and Ricardo Signes, and this only increased as I dug deeper and deeper on CPAN Testers' internals.


David and Barbie patiently explained to me all the bits and pieces I needed to know, and about how a lot of the things that should be common to all CPAN Tester's clients were currently scattered around, some even with different implementations, and how great it would be if all clients - including my new cpanminus one - could share all that from a since module. They wanted to do this since the QA Hackathon in 2009, but never got around to it.
...so I got a round tuit :-)
If we could pull this off, it would not only mean life getting much easier for the CPAN Testers' clients, but also that we'd finally be able to create a next generation system to take full advantage of the power of Metabase, sending even more information (and in a much more structured manner) to the infrastructure.

CPAN::Testers::Common::Client

This module aims to provide common utilities to CPAN Tester's clients, populating all Metabase::Facts and getting them ready to be sent via Test::Reporter. It even composes the legacy email for you with the data you provide. Code is already on github, and I'll push it to CPAN as soon as it's stable and ready for consumption.

App::cpantesters

As work on CPAN::Testers::Common::Client started to take shape, I was able to plug it into the build.log parser and see it come to life. Code is pretty raw at this point as I want to make sure it's doing its thing correctly before making an actual application out of it - and before sending data to the servers! If you want to help, I encourage you to try it out and tell me whether it worked or blew up. I can be found on irc.perl.org as "garu", or you can just file a bug report (in which case, please attach the unparsable build.log file if you can).

Here's how to do beta-test it:

1. download and install CPAN::Testers::Common::Client from github;
2. download App::cpantesters from github (no installation yet);
3. mkdir /tmp/reporter

After that, just use cpanm to install modules as you normally would. After each attempt, whether it passed or failed, go to App::cpantesters' base directory and do:

   perl -Ilib bin/cpantesters.pl

You should see some debugging info on the terminal, and after it's done you can see the reports it generated in /tmp/reporter. The files are plain text, so just open them in your favourite text editor and make sure everything seems ok!

HC SVNT DRACONES


The modules above are not ready for general public consumption by the time of this writing - which is why they write to disk instead of sending to CPAN Testers. I still need to coordinate with Andreas whether the email itself is parseable, and with him, Barbie, David and BinGOs as to adapt the CPAN Testers toolchain to use CTCC, send/receive the data from new Metabase Facts, and maybe extracting even more "moving parts" into CTCC, and now that the QA Hackathon is over it might take a bit more time. But it was already a great step forward!

All work and no play?

Every night after the hackathon we'd get together for dinner and hung out afterwards. I got a chance to see a 3D printer in action, play with the new Galaxy Tab, talk like a pirate (Arrrr! Thanks Wendy!), drink some Chartreuse, give a whole new meaning to my cleric's "turn undead" ability during a very cool Role Playing session, and travel from Stalingrad to Oberkampf (you had to be there).

I also learned a lot just by listening to some of the conversation that went by, and got a chance to talk to some really smart people about their very cool projects, some which I might even tackle in the near-ish future.

Oh, and yes. There were pictures =)

See you next year in the UK!