Table of Contents
  1. Mango 0.01000_12 Released
  2. Five a day, May 10
  3. Five a day, part 3
  4. (Five a day) x 2
  5. Catalyst book code
  6. Rose::HTML::Form and DBIx::Class
  7. Spring 2008 - a UKUUG Conference Review
  8. Catalyst - Accelerating Perl Web Application Development
  9. Writing Facebook apps in Catalyst
  10. Announcing Osgood: An Event Repository
  11. Hanging Antlers On Your ResultSets
  12. Ayalike: Content Management using Catalyst
  13. Refreshing a row from the DB - another note to myself
  14. Automatic DBIC schema creation from a database
  15. Mango 0.01000_10 Released
  16. Catalyst Cologne: My Wife FTW
  17. Introducing App::TemplateServer
  18. Mango 0.01000_07 Released
  19. New Year. New Progress.
  20. WoC part two: Database and Domain Design
  21. Vienna WoC TODO Manager series on shadowcat.co.uk
  22. A Christmas Catalyst Site: I Watch Soccer
  23. Update to 'Tags and search and DBIx::Class'
  24. Catalyst Book
  25. Have you ever forgotten where some subroutine comes from?
  26. Database Quoting with DBIx::Class
  27. Visualizing your DBIC schema
  28. Catalyst Tips: Part 1
  29. Catalyst::Controller::reCAPTCHA
  30. WIAB hits version 0.02
  31. Catalyst::Example::InstantCRUD updated to newer versions of underlying libraries
  32. Mango 0.01000_06 Released
  33. Database Abstraction - code vs infrastructure
  34. Catalyst and DBIx::Class: Part 1
  35. Sorry for the Delay
  36. DBIx-Class-QueryLog 1.0.0 Released
  37. High Availability using Catalyst & FastCGI external serv
  38. The Catalyst hosting web page
  39. Catalyst Integration: Day 99
  40. DBIx::Class - virtual columns?
  41. Fat Model
  42. Handel 1.0 Released
  43. Handel 0.99_19 Released
  44. Catalyst swag to hide your shame
  45. DBIx::Class 0.08001 released (and a podcast)
  46. Single Sign-On (SSO) with and without subdomains
  47. Perl, singletons, DAOs oh my!
  48. Catalyst Javascript Testbed
  49. Mango 0.01000_04 Released
  50. Catalyst Integration: Day 26
Mango 0.01000_12 Released
Mango Framework
I'm still plugging away. Mostly fixes for feeds and whole lot of live catalyst tests using Test::Class. I really really need to get checkout rolling. I just made it through a Google Checkout integration elsewhere and it would be nice to add that while it's still fresh in my head.
0.01000_12 Mon May 12 21:18:25 2008
    - Fixed various controllers to use Chained instead of index : Private
    - Converted Live Catalyst tests to Test::Class and added tests for path config option and subclass renames
    - Added RSS tests to live tests
    - Added Admin tests
0.01000_11 Sun Apr 13 20:39:27 2008
    - Added just enough pod to get spelling/coverage happy. Needs serious cleanup.
    - Added critic tests and set perltidy standard for dist
    - Refactor View::Feed. Needs more tests.
Five a day, May 10
Brian Cassidy
My latest five: RT #33158 - rejected (DBIx::Class) RT #25445 - already applied (DBIx::Class) RT #26978 - closed (DBIx::Class) RT #31473 - closed (DBIx::Class::Schema::Loader) RT #29041 - fixed this last year (Catalyst::Plugin::Session::PerUser)

Read and post comments | Send to a friend

Five a day, part 3
Brian Cassidy
I ventured into some DBIC related bugs this time. RT #32497 - rejected (Catalyst::Model::DBIC::Schema) RT #31848 - doc fixed (Catalyst::Model::DBIC::Schema) RT #29282 - duplicate (DBIx::Class::Schema::Loader) RT #23749 - already patched this last...

Read and post comments | Send to a friend

(Five a day) x 2
Brian Cassidy
I ended up doing twice my quota for today (it's easy to take care of the low-hanging fruit). A variety of Catalyst::* and DBIx::Class tickets were closed. RT #15941 - rejected (Catalyst::View::TT) RT #32254 - duplicate (Catalyst::Plugin::Authenti...

Read and post comments | Send to a friend

Catalyst book code
Jonathan Rockway

For a while, people have been requesting runnable code for the apps in my book. It has actually been available for a while, but apparently doesn't show up in Google results. It's because there is book called "Catalyst Code" that obscures results for the logical catalyst book code queries. Amusing.

Anyway, every application (in runnable form) is available here:

http://jrock.us/Catalyst-Book-Code-20080131.tar.gz

Enjoy.

BTW, I really don't think you're going to learn anything if you just cut-n-paste from the code. The best way to use the book is to actually read the words, think about them, and then try the examples. If you cut-n-paste your way through the book, you will have exhausted the resource without learning anything.

Finally, it seems some reviewers on Amazon are upset that the book is not a comprehensive manual of All Things Catalyst. It isn't, and I didn't intend for it to be. It's a tutorial so that you can follow along with, learn one way of doing things, get enough context to understand the online docs, and then use those from then on.

Catalyst is not really all that complex, and if you pretend that it is you're going to be upset. It is not a religion or way of life, it's just a module that makes gluing your application to the web easier. Nothing more.

I will agree that the book has some problems, however. One is that Chapter 3 sucks because of the $c->form (instead of $self->formbuilder) misprint. I didn't actually make that syntax up, as some reviewers contend, that's how Catalyst::Plugin::FormBuilder worked when I was writing the book. That is now deprecated, and most places in the book were changed to reflect that, but not all of them apparently. Sorry.

Anyway, my advice is to read the book even if you run into trouble in Chapter 3. The rest is much smoother, and you'll learn a lot.

Rose::HTML::Form and DBIx::Class
Zbigniew Lukasiak
Has anyone tried that combination?
Spring 2008 - a UKUUG Conference Review
Suretec
I got back last night, after a somewhat hectic Flybe.com flight (long story).

I really enjoyed the conference, my first time speaking at one, bit nervous, but it can only get better ;-)

Tuesday:

As I arrived around 10.30am on Tuesday, I just caught the end of Testing with Selenium, very handy bit of kit.

- System Configuration : An end to hacky scripts? This was very good and showed an alternative (IMHO, much better) to things like Cfengine and Puppet
- An Introduction to DBIx::Class - Tom Hukins is an excellent speaker and a very nice guy to boot. Had some good chat in the pub afterwards, mainly Perl related. He knows his stuff.
- Building a new model for account management - I've meet Simon a few times before, he's another great guy. His talk was about a completely new way to manage accounts (and much more) that they are using at Edinburgh uni.
- Software patents and open standards - This was one hour of very interesting, worrying and intelligent discussion.

It was good to see so much Catalyst usage too.

The rest of the day was spent chatting with Matt and Mike, meeting lots of very clever people, drinking, eating and generally have brilliant discussions over dinner and in the pubs ;-)

Wednesday:

I sat in the PostgreSQL talks all day.

Simon did an excellent job being the event host. Dave gave us an insight into how the Postgres project works..pretty much like other well organised Open Source projects of this size. Chris fewtrell talk was interesting to see how Postgres is used at the core of a business.

Explaining the Explain was very technical, but also fun. I think some useful things for Postgres will come out of the HA talk.

Mark Cave-Ayland was another excellent speaker and his talk on Integrating Map Data with PostGIS was well worth attending.

A real eye-opener was Simons talk on PostgreSQL Performance Features in 8.3. Well worth an upgrade!

Simon has written his thoughts on the event up too.

Gavin.

Catalyst - Accelerating Perl Web Application Development
Michele Beltrame
Catalyst Jonathan Rockway Packt Publishing, 2007 ISBN-13: 978-1-847190-95-6 US$ 39.99 - UK£ 24.99 Rating: 4/5 (very good) This is the first Catalyst-related book to be published, and I'm very happy of this as Catalyst is my platform of choice...
Writing Facebook apps in Catalyst
Zbigniew Lukasiak
A few weeks ago I've started a howto page for writing Facebook applications in Catalyst at the Catalyst wiki. Now I have accumulated there a few tips. Everyone using Catalyst for Facebook apps is invited to add their problems and solutions.
Announcing Osgood: An Event Repository
Cory 'G' Watson

Yesterday I released Osgood::Client and today, after realizing I had botched the upload, Osgood::Server. Unfortunately the documentation is a little thin, so I’ll take this opportunity to both inform the world and the module of it’s purpose.

Osgood is a passive, persistent, stateless event repository. The current docs say queue rather than repository but we’ve decided it’s a bit of a misnomer. A primer on the aforementioned explanation:

  • Passive: Osgood doesn’t seek out your events, it only waits for notification and query
  • Stateless: Querying an event does not change it
  • Repository: Once you’ve inserted an event it stays there forever

So what’s that mean? Osgood (::Server) is a system wherein you record the fact that something happened. A client library is provided that allows you to do just that (::Client). You can also query the server to ask it what has happened and it will inform you. I believe an example is in order.

Magazines.com’s backend is basically a big order pipeline after the order is taken. We are phasing out an AS/400 which is currently handling fulfillment of orders. If a customer calls us and wants to cancel a magazine, our our order entry system cancels the order and then needs to notify the AS/400. We have an audit table for order line items, so we would query that table for fresh cancels and send them to the 400 to finish the process.

All was well! Then our Marketing team approached us with a good idea: Send customers who’ve canceled an email enticing them back! It was a great idea until we released that we basically had one shot in our aforementioned setup to send a cancel, and we were using it. There was no way to keep up with the status. Had we sent it to the 400? What about to our email service provider? Were there other things we might do in the future? Osgood was born.

Osgood is based around Events. You use ::Client to send an event to the Osgood server, like so:

my $event = new Osgood::Event(
	object => 'Moose',
	action => 'farted',
	date_occurred => DateTime->now(),
	params => {
		name => 'Hector'
	}
);

Events are composed of an object, an action and a time. These fields are all stored as VARCHAR(64)s, so you can put anything you like up to that size.

The optional params accepts a hashref of name value pairs. This allows storage of information that doesn’t fit Osgood’s Object/Event setup, but that you might need when using the event. A good example would be the primary key of the specific Moose that farted. Note that you can also use get_param and set_param on an Event object.

my $list = new Osgood::EventList(events => [ $event ])
my $client = new Osgood::Client(
	url => 'http://localhost',
	list => $list
);
my $retval = $client->send();
if($list->size() == $retval) {
	print "Success :)\n";
} else {
	print "Failure :(\n";
}

You can put as many events into an EventList as you like. Then you create a Client and send it! Above we check that send returned a number that matches the events we sent.

That’s all well and good, but how do you use what you’ve stored? The client also works the other way around:

my $retval = $client->query({
     object => 'Moose',
     action => 'farted',
});
 
if($retval) {
	# Gets an EventList
	my $list = $client->list();
	my $iterator = $list->iterator();
	while($iterator->has_next()) {
		my $event = $iterator->next();
		print $event->get_param('name')." farted at ".$event->date_occurred()."\n";
	}
} else {
	print "Oh noes!\n";
}

We asked Osgood for all the Moose/farted events and we got them back. One note of interest is that Events are assigned an ID when they are stored, and that ID is present when you get them in a query. This will come in useful later.

The query method supports many other options as of this writing:

  • date_after: date after the specified one
  • date_before: date before the specified one
  • id: all events with an id greater than the one specified
  • limit: only return the specified number of events

So back to our original problem. Rather than make our internal cancel process complex, we simply create a row in a queue table which is emptied frequently and sent to Osgood. Our backend processing jobs then query Osgood using the id parameter to query to find every event that was created since the last one it processed. Now we can create an unlimited number of backend jobs that work from the same event. Our email sending and publisher notification of cancels happen independently.

This work was inspired by past work with Publisher/Subscriber systems and message queues like the venerable JMS. It bears very little resemblance to those ideas, but one might be able to squint a bit and see the similarity.

Osgood::Server is a Catalyst application using the XML::REST plugin and ::Client is Moose based with some help from MooseX::Iterator and some XML::XPath and XML::DOM magic for serializing and deserializing the eventlist.

If you have any questions, feel free to drop me an email or hit me up on irc.perl.org as gphat.

Hanging Antlers On Your ResultSets
Cory 'G' Watson

At some point in the not too distant future, the intarweb’s magazine consumers will be perusing a new magazines.com. I’ve got the responsibility of building it. One of the requirements is to allow our Merchandising and Marketing folk to get their jobs done without too much fuss. The system they use will consume sets of products. They might want to build a set of magazines in a certain category, under a certain price. What kind of tool am I talking about?

A Query Builder.

I can hear your eyes rolling already. We’ve all written them. But I’m going to share an extra-cool method for doing so.

Here’s the list of ingredients you’ll need.

Now that you’ve collected your ingredients, let’s prepare them.

First, note that we aren’t allowing our end users to build literal queries in this recipe. We are going to allow them to chain calls to our resultset. This is a much better option, in my opinion. A lot of the complexity can be removed by simplifying some heinous or dangerous SQL into a simple resultset method. With the help of Moose we will create a custom resultset that allows us to introspect the methods available for building and show them to the user automatically.

Now, we’ll be working on a Product example. Our end user needs to build a query that finds all the products in a price range, of a certain type and in a certain category.

Since you’re already versed in custom resultsets I’ll just show you the finished product:

package Our::ResultSet;
use Moose;
use MooseX::Method;
use MooseX::Util::TypeConstraints;
 
extends 'DBIx::Class::ResultSet';
 
subtype 'Currency'
	=> as 'Object'
	=> where { $_->isa('Math::Currency') };
 
coerce 'Currency'
	=> from 'Num'
		=> via { Math::Currency->new($_) };
 
method price => named (
	operator => { isa => 'Comparator', required => 1 },
	amount	=> { isa => 'Currency', required => 1, coerce => 1 }
) => sub {
	my ($self, $args) = @_;
 
	if($args->{'operator'} eq '=') {
		return $self->search({ price => $args->{'amount'}->as_float() });
	}
 
	return $self->search({
		price => {
			$args->{'operator'} => $args->{'amount'}->as_float()
		}
	});
};

That’s a lot of code, but the concept is very simple: Create your resultset’s methods with MooseX::Method and then you can do the following magic:

my $rs = $schema->resultset('Product');
my $map = $rs->meta->get_method_map();
foreach my $method (keys(%{ $map })) {
	# See MooseX::Method docs for what to do with this.
	my $signature = $method->signature();
	# Profit!
}

Having the method’s ’signature’ allows you to get a list of the parameters that the method expects, as well as the types and ‘required’ status. That sounds like the recipe for some go-getting developer to construct a query building tool!

One warning. MooseX::Method currently lacks a permanent API for acquiring this information. The current API is to call export on the signature. You can Dumper the resulting hashref and see pretty easily how to get what you need. Keep an eye on The CPAN or in irc.perl.org’s #moose for updates.

So to use this recipe, you’ll have to construct your own mechanism for taking the above information and generating a UI for your application. I got my Catalyst based query-builder working this afternoon.

If you have any questions feel free to drop into irc.perl.org and shoot a message to gphat. You can find me in #moose.

Ayalike: Content Management using Catalyst
Cory 'G' Watson

UPDATE: The demo and trac links are down atm, as Ayalike is being moved to new hosting.

I’ve been hacking away for the last week or so on a content management system (henceforth CMS). We are going to need a CMS for some future $work projects and I’m generally unhappy with all that I’ve reviewed. So many of the existing ’solutions’ seem to be more in the business of providing you with a ‘platform’. Most of the folks I know build platforms for a living, we need only find a way to manage the content.

So with that in mind, Ayalike (pronounced like ‘a uh like’) aims to provide the following features:

  • Multisite, multiuser, multirole
  • Output agnostic
  • Versioning
  • Pluggable publishing
  • Output processing

Three of those features are complete. Publishing works, but is currently not pluggable. Output processing is there as well, but I’ve yet to hook it into anything but unit tests.

That’s all kind of abstract, here’s what I’ve done in a more concrete form: This week I modified one of our internal applications to use a custom Template Toolkit provider that communicated with Ayalike and provided ‘preview’ functionality on the live site without changes to the application itself. The application functioned normally, rendering compiled templates from the filesystem, unless given a special parameter. In that case it requested the entry from Ayalike over HTTP using it’s wiki-esque page viewing feature. Obviously real apps would protect this feature with logins.

I’m not quite ready to tout it on the mailing lists or in IRC yet, but I’m very excited about the project and I hope that it might become the solution for my future work projects. Regardless, Catalyst needs a CMS.

Are you interested? Visit the Trac site. There’s even a working demo, cleared every half hour.

Refreshing a row from the DB - another note to myself
Zbigniew Lukasiak
This is another thing that I keep forgetting - how to refresh a row with values from the database.
So here it is:

$row->discard_changes;

It comes from DBIx::Class::PK (and not from DBIx::Class::Row where I usually start the serach).

Update: The core devs say that the new version of documentation will mention discard_changes in DBIx::Class::Row POD. Thanks.
Automatic DBIC schema creation from a database
Zbigniew Lukasiak
This is something that I keep forgetting and then I have to wade through the docs again. And there are traps in the docs - for example SQLT can do the schema creation - but it does not build the relationships.

So here is the magic invocation for DBIx::Class::Schema::Loader which do create the relationships (if you use the right switch):

perl -MDBIx::Class::Schema::Loader=make_schema_at,dump_to_dir:./lib -e make_schema_at("New::Schema::Name", { relationships => 1 }, [ "dbi:Pg:dbname=foo","postgres" ])’
Mango 0.01000_10 Released
Mango Framework
You know the routine. Release [crap] early. Release [crap] often. It's going to be a while till the next update.The Catalyst checkout controller code is going to eat my brain for most of the weekend. After that, it's onward to refactoring, cleaning up templates and adding a buttload of fluff columns to the schema.
0.01000_10 Fri Jan 18 22:27:19 2008
    - Added Atom/RSS Feeds to Wishlists
    - Added Atom/RSS Feeds to Products(tags)
    - Added Feed() action attribute and enable_(feeds|atom|rss)_feed
    - Fixed REST->entity() to use new config
    - Stop importing blessed in Feed View
Catalyst Cologne: My Wife FTW
Jay Shirley

Catalyst Cologne

My wife just returned from shopping and giddily approached me and asked, "How do you spell Catalyst?" After confirming the spelling, she pulls out a box and goes, "It's Catalyst... Cologne!" I love her for a) knowing the project and b) identifyin...

Read and post comments | Send to a friend

Introducing App::TemplateServer
Jonathan Rockway

Sometimes you want to be able to view rendered HTML templates without having a big Apache or Catalyst application do it. You might want to play with a fun new engine you heard about, or perhaps you want to give your web designer the templates and a renderer to work on while on a plane.

Well, now you can install App::TemplateServer from the CPAN and do just this.

Let's say you have a TT template like this in /myapp/templates:

[%# foo.tt -%]
Hello, [% there %]!

To see this rendered with App::TemplateServer, just create a YAML file containing your variables:

---
there: world

Then start up a server:

template-server --docroot /myapp/templates --data vars.yml

Then browse to http://localhost:4000/. There you'll see a listing of the available templates. Click through to http://localhost:4000/foo.tt, and you'll see:

Hello, world!

You can obviously use more complex templates than this. The data file parser uses Package::FromData to create packages with mocked methods and class data for you, so you can easily emulate Catalyst or Jifty environments and give the templates some good test data to work with.

Right now there are TT, Mason, HTML::Template, and TD backends on the CPAN. Why not write one for your favorite templating system? It's only a few lines of code:

package MyProvider;
use Moose;
with 'App::TemplateServer::Provider::Filesystem';
   

sub render_template { My::Engine->new->render($_[0], data => $_[1]->data) }
1;

Enjoy.

Mango 0.01000_07 Released
Mango Framework
It's ugly. It's incomplete. It sucks. But a release is a release and progess is progress. My only goal for a while is to keep myself from bogging down in details and thoughts of refactoring and just get things functional. A store in a box has a buttload of pieces to it (user pages, public pages, admin pages, core, etc) and Mango isn't even a one-solution-fits-all-does-everything type of app. But I digress. It seems that going back to writing live (Test::WWW::Mech) tests seems to better spur productivity, tests I will most definitely need later when refactoring.
0.01000_07 Mon Jan 7 19:07:23 2008
    - Added Catalyst::View::TT/Session/Cookie/Store to PREREQ
    - Fixed problem in tests with latest Error.pm
    - Updated XML::Atom PREREQ and test
    - Mango->share now uses module_dir and falls back to ../../share when using
        local INC files
    - Added mango.pl to create app/config/data/components/database
    - Moved users specific controller/form/templates into Users/users namespace
    - Fixed urls to deal with not ending in /
    - Updating an Attribute wasn't setting update()
    - Converted admin to RESTish access
    - Controllers now inherit from M::Catalyst:Controller (which does REST/Form
    - Merge plugins into Plugin::Application
    - Added Auth helpers is_admin/unauthorized
    - Added REST helpers want_html/browser/yaml/feed, etc
    - Form I18N is now FIELD_LABEL_$FIELD instead of LABEL_$FIELD
    - Form I18N is now BUTTON_LABEL_$FIELD instead of LABEL_$FIELD
    - Form I18N is now CONSTRAINT_$CONSTRAINT instead of $FIELD_$CONTSTRAINT
    - Wishlist admin works mostly, with tests
New Year. New Progress.
Mango Framework
Time to get rolling again. I started writing 'live' (Test::WWW::Mechanize::Catalyst) tests for force myself to focus on the web side of things and get my nose out of the base class details. I was bogging myself down with all of the things to remember and the possibilities of re factoring. Spent some time patching Catalyst::Action::REST to deal with some issues that were biting me in the ass. My goal this week is to get an _07 out the door by the end of the weekend that at least has a working quickstart, cart, wishlists and the admin stuff tested. Still way behind. :-(
WoC part two: Database and Domain Design
Matt S Trout
Part two of the Vienna WoC TODO Manager code is up, in which I design and implement a Catalyst+DBIx::Class+Reaction app from the ground up in public and document the process. The latest article is Database and Domain Design, in which I don my gea...

Read and post comments | Send to a friend

Vienna WoC TODO Manager series on shadowcat.co.uk
Matt S Trout
I'm going to be building an application to help run the Vienna.pm Winter of Code, and since the resulting application is going to be open source and in public subversion as an example of Catalyst+DBIx::Class+Reaction, I've decided to write up the ...

Read and post comments | Send to a friend

A Christmas Catalyst Site: I Watch Soccer
Jay Shirley
I was very saddened to see a posting on SoccerTV.com that they're closing up shop after selling their assets. It's a very nice and easy way to just view what upcoming games are going to be on. I only get Fox Soccer Channel, so it isn't that big ...

Read and post comments | Send to a friend

Update to 'Tags and search and DBIx::Class'
Zbigniew Lukasiak
Advanced Search in web DBIx::Class based applications (with tags, full text search and searching by location) is an update to Tags and search and DBIx::Class.

I think it is a nice technique.
Catalyst Book
Jonathan Rockway

My Catalyst book is out and is beginning to appear in the hands of fellow Catalyst users! You can buy it from Amazon and local book stores, but it looks like the publisher (Packt) has the best price right now.

Basically, this book covers getting started with Catalyst. The goal is to take you from "what's MVC?" to being able to write complete web applications with Catalyst. We'll build a few complete applications including an address book, a mini-blog, and ChatStat (which is basically Ircxory but with a Template Toolkit interface instead of one built with Template::Declare).

Along the way you'll learn all about Catalyst, DBIx::Class, and the Template Toolkit. I cover REST interfaces, AJAX (with Jemplate), RSS, testing, deployment, and more. The book is very hands-on oriented; I usually explain what sort of code I want to write, write the code, and then explain what the code does. If necessary, we look at theoretical examples of when related features would be useful, and then move on to the next piece of code.

The full PR blurb is on the book's website, so please take a look.

Have you ever forgotten where some subroutine comes from?
Zbigniew Lukasiak
Or have you ever been learning a big stack of modules, like DBIx::Class or Catalyst and stumble from time to time over a new method name and you did not know where to look for it's documentation? I have once even wrote a script that generated an index of all methods documented in some code directory tree. It worked for me when learning DBIx::Class - but you needed to be very careful when choosing what you are indexing - you indexed too much and the list just got too long, you indexed too little and it's coverage was too limited.

Until discovering Pod::POM::Web I did not know that what I need is actually a full text search over all the Pod documents - you cut and paste the unknown method name into the search box press Enter and voila you have list of modules that document that method. And if you know that it comes from one of the DBIx::Class submodules you just add DBIx::Class to the search criteria - and don't get all the Class::DBI documentation pages. No problem with too long index lists. Someone should hack it to run against a full CPAN mirror and beat the other CPAN search websites.
Database Quoting with DBIx::Class
zzo
I was having insane problems trying to CONCAT text during a database UPDATE to a text field. I want to use the database's (mysql in my case) CONCAT function because the data could be large & only wanted to pass in new lines & not the entire contents of the column.

So for a column called output I would CONCAT to it like so:

$row->update( { output => \"CONCAT(output, $new_stuff)" } );

Which is groovy as long as $new_stuff didn't contain single quotes or question marks. Single quotes throw off database quoting and question marks throw off DBI...

So there's a 'quote' function available via DBI which takes care of the single quotes - usually - but completely ignores question marks. Look at it in DBI.pm - it's pretty, um, bare-bones.

So I would call DBI's quote function & then use an ugly regex to prepend all question mark's that didn't already begin with a backslash with a backslash - and this worked - mostly.

There were several problem that would crop up intermittently & of course always at the exact wrong time. If a single quote was already preceded by a backslash calling DBI's quote made it worse & the update would fail (note DBI's quote function replaces all single quotes with 2 single quotes & then puts single quotes around the entire string.)

I also had problem with my regex seemingly not working in all cases to properly hide the question marks from DBI & would get errors occasionally about providing too many or too few parameters for the given number of question marks.

Welp after just dealing with the errors that would occasionally pop up I finally got off my butt & came up with a quoting fix that so far as been 100% reliable for any combinations of single quotes, question marks, & backslashes before them.

The secret is to totally get rid of those character & replace them with char(39) and char(63) for single quote and question mark respectively.

The deal is to delete any backslashes before any single quote or question marks in the output, the replace those character with their 'char' equivalents in-between single quotes and commas (as CONCAT can take multiple values).

Here's what it looks like:

sub concat_db {
my($row, $column, $value) = @_;

# get rid of an backshlased single quotes
$value =~ s/\\'/'/g;

# clean up single quotes
$value =~ s/'/', char(39), '/g;

# clean up '?'s
$value =~ s/\\\?/?/g;
$value =~ s/\?/', char(63), '/g;

# concat it
$row->update({ $column => \"CONCAT($column, '$value')" });
}

I put this code in my top-level DBIC::Schema class & call it thusly:

MYDB::concat_db($row, 'output', "My wacky \'output\' ??? 'yoyo' ");

& magically the correct quoting happens.

Here's what that string looks like after its been quoted:

'My wacky ', char(39), 'output', char(39), ' ', char(63), '', char(63), '', char(63), ' ', char(39), 'yoyo', char(39), ''

Note the first & last single quote get put there in the CONCAT statement itself.

Happy Quoting!!

Visualizing your DBIC schema
Jonathan Rockway

If you want a somewhat pretty picture of your DBIC schema (with relationships drawn, of course), install GraphViz, SQL::Translator, and DBICx::Deploy from the CPAN, and then run:

$ dbicdeploy -Ilib MyApp::Schema ~/graphs GraphViz

~/graphs will then contain a .sql file that is actually a png of your schema. Rename it and see your schema in your favorite png viewing application.

Oh yeah, the post needs to contain the word "Catalyst" so that Planet Catalyst will pick it up. Catalyst. :)

Catalyst Tips: Part 1
Jonathan Rockway

I haven't had much to blog about recently, but here are a few tidbits you might enjoy while I think of something more interesting to write about.

Component->config is write-only

People seem to forget this all the time. Good use of config:

package MyApp::Model::Something;
use base 'Catalyst::Model';
__PACKAGE__->config( static => 'data' );

Bad use of config:

package MyApp::Model::Something;
...
sub method { 
    my $self = shift;
    $self->config->{some_key}; # XXX
}

You can't read from config. Everything in config gets merged into $self at COMPONENT time:

sub method {
    my $self = shift;
    $self->{some_key}; # correct
}

Note that there are many ways to get configuration into a component. You saw one, __PACKAGE__->config( key => value, ... ) above.

Some others:

package MyApp;
use Catalyst;

__PACKAGE__->config->{Model::Something} = { some_key => 'some value' };

If you use ConfigLoader with YAML, you can do this too:

---
Model::Something:
  some_key: 'some value'

TMTOWTDI. I generally don't put anything critical in the config file, though, because I don't want users to touch that stuff.

Common configuration

If you have an app that consists of more than just the Catalyst part, you might find sharing a ConfigLoader-loaded config between the components troublesome. The solution is to write a config class like:

package MyApp::Config

... load config ...

sub get_catalyst_config {
   my ($class) = @_;
   return %{$class->{config}}; # or whatever
}

Then you can easily use it in Catalyst:

package MyApp::Web;
use MyApp::Config;
use Catalyst qw(...);

__PACKAGE__->config( MyApp::Config->get_catalyst_config );

This technique was suggested to me by Dave Rolsky after I released Catalyst::Plugin::HashClass, which replaces $c->config (and $c->stash, if requested) with a class of your choice.

Since $c->config is read-only after setup, HashClass (for config) isn't all that useful. Feel free to use it if you want to add accessors to $c->config, though. (So you can write $c->config->foo instead of $c->config->{foo}.)

Deploying your DBIC Schema

DBIx::Class lets you deploy your DBIC schema to a database:

use My::Schema;
My::Schema->connect('DBI:...')->deploy;

That's a lot of code to write, though, so there is a module called DBICx::Deploy on the CPAN that includes a script called dbicdeploy. If you have that, you can just say:

dbicdeploy -Ilib My::Schema DBI:whatever:database

That's much quicker than writing a script. You can also generate files containing the necessary SQL:

dbicdeploy -Ilib My::Schema my_schema

Convenient.

Testing your DBIC Schema

Using a database in unit tests used to be painful, but with DBIC it's a breeze. Install DBICx::TestDatabase from the CPAN, and then you can:

use Test::More tests => ...; 
use DBICx::TestDatabase; 
my $schema = DBICx::TestDatabase->new('My::Schema'); 

Now $schema is a fresh My::Schema you can use. Behind the scenes, DBICx::TestDatabase creates a temporary SQLite database, deploys to it, and returns the connection. It handles cleanup for you, so you don't have to do anything. Just use the database as you see fit, and it will go away when you exit.

Bash alias for running the Catalyst server

I find myself switching between the 6 or so Cat apps I maintain fairly frequently, so perl script/myapp_server.pl never seems to be nearby in history. To combat this problem, I have a bash alias that looks like:

alias cs='perl script/*_server.pl'

Then I can just type cs -r -d anywhere and get a catalyst server for the app I'm working on.

Anyway, that's all for tonight. Have fun.

Catalyst::Controller::reCAPTCHA
Kieren Diment
Just a quick note to let you all know that I made a Catalyst extension to interact with the reCAPTCHA service.

It's Catalyst::Controller::reCAPTCHA on cpan and comes with documentation and a simple example application. If anyone wants to add patches for Captcha::reCAPTCHA::Mailhide api, feel free :-)

By the way, watch out for testing this on your dev box - I got myself banned from reCaptcha for several hours by posting continually failing recaptchas.

WIAB hits version 0.02
Kieren Diment
WIAB - Website in a box, my simple Catalyst CMS has hit version 0.02. The main change here is that we're using some Jquery code to make editing pages easier. The plugin in question is JTagEditor. You can download the new version from here.
Catalyst::Example::InstantCRUD updated to newer versions of underlying libraries
Zbigniew Lukasiak
Version 0.0.20 released to CPAN. My plan is to port it to use the new HTML::FormFu instead of HTML::Widget and then perhaps make a web wizard to edit the config options.
Mango 0.01000_06 Released
Mango Framework
Back on the wagon again.
  • Fixed Template->share when Mango has never been installed
  • Stripped action from yml forms, now get path_prefix on the fly
  • Base Controllers have form_directory set to cope with subclass rename
  • Cart Controller works with add/update/clear/save/delete
  • Added Forms plugin companion to Forms controller
  • Cart forms is not build from sub-forms
  • Form->render now localizes field labels/submit
  • Views now fall back to auto/Mango/templates
  • Added Form/FormFile attributes to Form controller
In general, I think I finally have things the way I want them in terms of forms, base controllers and the various plugins and custom authentication. Next step is wishlist view/restore support and merging the various plugins into on [thing to load] Mango::Catalyst::Plugin::Application.
Database Abstraction - code vs infrastructure
John Wang

I've worked on a number of database-driven projects and no matter how much people want database abstraction, it was always difficult to code and maintain. I was recently reminded of this when I read this Drupal article on dropping PostgreSQL support. Not only can it be difficult to maintain support for multiple databases, but it may be difficult to find developers.

One solution of modern programming is to move database abstraction from the code to the infrastructure using a ORM (Object-Relational Mapper) or Data Mapper. A ORM and Data Mapper abstracts the database for you so you no longer have to do tie db abstraction to each app. Not only does it let you code once for multiple databases it lets your users migrate their data from one database to another. This blog runs Typo which is based on Ruby on Rails and ActiveRecord. I've been contemplating migrating Typo from MySQL to PostgreSQL and I've been told that it would be as simple as exporting the data with YAML, updating the database.yml file and importing the data. I haven't gotten around to doing it yet but it is a powerful idea. ActiveRecord is a data mapper and isn't as flexible as a full blown ORM but it gets the job done for the most part. For a full-blown ORM, I think of Perl's DBIx::Class which provides a full OO interface to the RDBMS allowing you to code just once for multiple DBs without limiting you when you want to use some esoteric database-specific SQL. DBIx::Class is often used with the Catalyst Framework but is also used by itself.

There are PHP frameworks out there like Symfony and Cake but do any of them have stand-alone ORMs? If so, could Drupal move to something like that and solve their maintainership problems once and for all? Drupal is part of the Go PHP5 effort so there should be no issue using PHP 5 OO. Something to think about for the Drupal folks if a PHP ORM is available.

Catalyst and DBIx::Class: Part 1
Chris Humphries
[For instructions on the installation of Catalyst, please see the Catalyst Manual page on installation.]

This is an attempt at introducing DBIx::Class and using it in Catalyst, so you can be familiar with it enough to start using it.

Understanding ORM



First, you'll need to understand the purpose of Object-relational mapping. It is what DBIx::Class is. To quote the wikipedia.org page:
Object-Relational mapping (aka O/RM, ORM, and O/R mapping) is a programming technique for converting data between incompatible type systems in databases and Object-oriented programming languages. This creates, in effect, a "virtual object database" which can be used from within the programming language.


So, what you end up with is classes that have the ability to look and feel like database tables, yet all read/write operations happen behind the scenes. This enables you to work with the database without having to make database specific calls. This also allows you to code where all you need to worry about in the code is just what to call, not what database is under the hood. DBIx::Class makes this easy and is pretty simple to wrap your brain around.

The basic idea is that you have one controller master class, and then one or more different classes that usually represent each table in the database. The controller master class is where you will specify the database connection information (and other optional settings) and load the table classes.

This may seem complex, yet it makes life much easier and allow your code to be much more flexibly to change. It also fulfills the need of having the Controller layer not be aware of the Model layer details in the Model-View-Controller design pattern (which is most common in serious web applications).

Getting to know DBIx::Class



Please take a moment to go over the examples in the DBIx::Class CPAN page.

Then go over the manuals (they're pretty brief and include code to explain everything): Intro, Example, Joining, Cookbook with interest on prefetch and joins, and Troubleshooting.

These documents do a much better job at explaining how to use DBIx::Class and all it's specifics than I could do. Be sure to check out the documentation map also.

Using DBIx::Class in Catalyst



Please table a moment to go over the Catalyst manual tutorial, the database access with DBIx::Class section. This will get you familiar with the basics.

If you already have an existing database schema and it's in a database, you can have Catalyst create them for you. It's advised that you just do this once, so that you can save some typing initially (if you already have a database with schema in it... like in this example). First make sure you're in your Catalyst root directory, made with this:
$ catalyst.pl MyTestApp

Then you can use the Catalyst create script to create your DBIx::Class classes for you:

$ cd MyTestApp
$ ./script/mytestapp_create.pl model DB DBIC::Schema DB::Schema create=static dbi:Pg:dbname=ticketingsystem chris
exists "/home/chris/MyTestApp/script/../lib/MyTestApp/Model"
exists "/home/chris/MyTestApp/script/../t"
Dumping manual schema for DB::Schema to directory /home/chris/MyTestApp/script/../lib ...
Schema dump completed.
created "/home/chris/MyTestApp/script/../lib/MyTestApp/Model/DB.pm"
created "/home/chris/MyTestApp/script/../t/model_DB.t"
$


Ok, now let's take a look at what it created:

$ head -13 lib/MyTestApp/Model/DB.pm
package MyTestApp::Model::DB;

use strict;
use base 'Catalyst::Model::DBIC::Schema';

__PACKAGE__->config(
schema_class => 'DB::Schema',
connect_info => [
'dbi:Pg:dbname=ticketingsystem',
'chris',

],
);
$


Ok, so from looking at the Model class for this database, it seems pretty easy to figure out. It creates a class that is based off Catalyst::Model::DBIC::Schema and sets up some configuration values (what the master DBIx::Class is and the database connection information used by DBI). Ok, now lets look at the DB::Schema file:

$ cat lib/DB/Schema.pm
package DB::Schema;

# Created by DBIx::Class::Schema::Loader v0.03009 @ 2007-09-04 19:26:55

use strict;
use warnings;

use base 'DBIx::Class::Schema';

__PACKAGE__->load_classes;

1;

$


Hrm, even simplier right? '__PACKAGE__->load_classes' subroutine loads up all the configured schema classes in the lib/DB/Schema/ directory. Let's look at one:

$ cat lib/DB/Schema/Status.pm
package DB::Schema::Status;

# Created by DBIx::Class::Schema::Loader v0.03009 @ 2007-09-04 19:26:55

use strict;
use warnings;

use base 'DBIx::Class';

__PACKAGE__->load_components("PK::Auto", "Core");
__PACKAGE__->table("status");
__PACKAGE__->add_columns(
"id",
{
data_type => "integer",
default_value => "nextval('status_id_seq'::regclass)",
is_nullable => 0,
size => 4,
},
"name",
{
data_type => "character varying",
default_value => undef,
is_nullable => 0,
size => 40,
},
);
__PACKAGE__->set_primary_key("id");
__PACKAGE__->has_many(
"users_ticket_status_logs",
"DB::Schema::UsersTicketStatusLog",
{ "foreign.status_id" => "self.id" },
);
__PACKAGE__->has_many(
"staff_ticket_status_logs",
"DB::Schema::StaffTicketStatusLog",
{ "foreign.status_id" => "self.id" },
);
__PACKAGE__->has_many(
"tickets",
"DB::Schema::Ticket",
{ "foreign.status_id" => "self.id" },
);

1;


This was obviously generated from PostgreSQL, as you can note the default_value for "id". Anyways, without focusing too much on that, the important parts are all the function calls ('has_many', 'set_primary_key', 'add_columns', 'table', and 'load_components'). Here is a breakdown on what they all are for:

  • load_components: Loads the components you need, which should almost always be the same (PK::Auto and Core) unless you need custom components detailed in the DBIx::Class component manual. [technically I guess you don't need PK::Auto as it's in Core now]

  • table: obviously sets the name of the table

  • add_columns: defines the columns of the table

  • set_primary_key: sets the primary key of the table

  • has_many: one of a few relationship definition functions for the table, see more about them in detail here.


It's pretty simple once you read the documentation links I gave. The naming of everything is pretty intuitive, so makes wrapping your brain around what's going on pretty easy.

Summary



So you should now have an idea of what DBIx::Class' purpose is and some details about it's use. You should also know where to look for documentation on each part discussed here and have a good general knowledge of DBIx::Class' implementation of ORM.

You should have a simple understanding of DBIx::Class and Catalyst. I'll continue in another article on more details and dive more into using DBIx::Class in Catalyst Controllers and using multiple databases.

Hopefully this was useful to you. I'd like to keep going, yet it's more appropriate to break this up into parts. If I made any mistakes or feel I should have written something in a better way, let me know and I'll be sure to update!

Enjoy playing with Catalyst and DBIx::Class!
Sorry for the Delay
Chris Humphries
Been busy with life and cramming knowledge into my brain lately, this took a backseat. Noticed RSS feed followers decline the longer I do not post, 2-3 days hopefully will be the happy medium.

Will get back on track tonight, starting with a series on Catalyst!
DBIx-Class-QueryLog 1.0.0 Released
Cory 'G' Watson

I just uploaded the 1.0.0 release of DBIx::Class::QueryLog to the CPAN.

The 1.0.0 label is mostly just a not to it’s maturity. I’ve been using QueryLog extensively at $job — most recently to tune a multi-million order conversion script — and a 1.0 is in order.

The only new feature is a refactoring of the ‘analysis’ code into a seperate object, as well as a new mode that totals up queries, combinining same queries and keeping a count and total time for all executions.

Have fun!

High Availability using Catalyst & FastCGI external serv
zzo
Zero Downtime / High Availability with Catalyst & FastCGI external servers
Intro

Here's the idea - you simultaneously run 2 FastCgiExternalServer's - a production one & a staging one. The staging one you muck around with to your heart's content. Start/stop/restart it - whatever - it won't effect your production environment. At some point you want to promote the staging stuff into production. And of course you cannot have ANY downtime.

I'm going to use an application name of 'mt' for this example - so replace it (or not) with your own - same for 'example.com'
All of these examples run fine on CentOS
You're probably going to have to change some paths too

Get Started

So you need 5 files - 2 httpd.conf's, 2 start/stop/restart fastcgi scripts, & 1 switchover script

    * The 2 httpd configuration files are a 'production one' - which you're running 99% of the time, and a 'temporary one' - that you use to gracefully promote your staging stuff into production.
    * The 2 start/stop/restart scripts are used to independently start/stop/restart your fastcgi external servers.
    * The 1 switchover script will actually do the promotion with zero downtime. When you're done promoting your production environment will exactly match your staging environment.

   1. create 2 httpd.conf's - 'httpd_prod.conf' & 'httpd_temp.conf' - each with a staging & production virtual server:

         1. http_prod.conf:

            <VirtualHost *:80>
                ServerName stage.example.com
                ErrorLog logs/example.com/stage_error_log
                CustomLog logs/example.com/stage_access_log combined
                Alias / /tmp/mt.stage/
            </VirtualHost>

            <VirtualHost *:80>
                ServerName example.com
                ServerAlias www.example.com
                ErrorLog logs/example.com/error_log
                CustomLog logs/example.com/access_log combined
                Alias / /tmp/mt.prod/
            </VirtualHost>

            # the staging socket
            FastCgiExternalServer /tmp/mt.stage -socket /tmp/mt.stage.socket

            # the production socket
            FastCgiExternalServer /tmp/mt.prod -socket /tmp/mt.prod.socket

         2. http_temp.conf: (so named because this is a temporary state when you want to make staging = production)
            Note the only diff is there's only 1 external server (the staging one) & the production virtual host now points to it

            <VirtualHost *:80>
                ServerName stage.example.com
                ErrorLog logs/example.com/stage_error_log
                CustomLog logs/example.com/stage_access_log combined
                Alias / /tmp/mt.stage/
            </VirtualHost>

            <VirtualHost *:80>
                ServerName example.com
                ServerAlias www.example.com
                ErrorLog logs/example.com/error_log
                CustomLog logs/example.com/access_log combined
                Alias / /tmp/mt.stage/

            </VirtualHost>

            FastCgiExternalServer /tmp/mt.stage -socket /tmp/mt.stage.socket

   2. You need 2 /etc/init.d (or whatever) scripts that start & stop your fastcgi servers: mt.prod & mt.stage - the differences are mt.prod uses /tmp/mt.prod & mt.stage uses /tmp/mt.stage for their sockets & they have different pid files (also I don't start as many processes for my staging daemons) - finally REALLY make sure PROD starts up before moving on.
         1. mt.prod:

            #!/bin/sh

            APP_PATH=<path to yer Catalyst base directory>
            FCGI_SOCKET_PATH=/tmp/mt.prod.socket
            PID_PATH=/tmp/mt.prod.pid

            case $1 in
                start)
                    echo -n "Starting PROD MT: mt_fastcgi.pl"
                    cd $APP_PATH
                    script/mt_fastcgi.pl -l $FCGI_SOCKET_PATH -p $PID_PATH -d -n 5
                    echo

                    # make real sure it's started
                    PID=`cat $PID_PATH`
                    if [ -n "$PID" ]
                    then
                        echo "Started"
                    else
                        echo "Start failed - trying again"
                        unlink $FCGI_SOCKET_PATH
                        $0 start
                    fi

                    ;;

                stop)
                    echo -n "Stopping PROD MT: "
                    PID=`cat $PID_PATH`
                    if [ -n "$PID" ]
                    then
                        echo -n kill $PID
                        kill $PID
                        echo
                    unlink $FCGI_SOCKET_PATH
                    else
                        echo MT not running
                    fi
                    ;;

                restart|force-reload)
                $0 stop
                sleep 10
                $0 start
                    ;;

                *)
                    echo "Usage: /et