Installing Parseexcel

Parseexcel is a ruby port of the perl parseexcel module.

It's installable via a nice gem like so:

That's that, now remember since it's a gem library we have to tell our script to use the gem libs when we run the script from the console using the -rubygems switch.

Using Parseexcel

Parseexcel is a very straight forward library, you can't do too much with it, but it gets the job done.

Getting a Workbook

This returns the actual excel file's workbook, from there we need to determine what worksheet we're on.

Getting a Worksheet

Will return the first worksheet, you could also use the each method on the workbook to iterate over all the worksheets.

Iterating over rows and columns

The worksheet object has a very nice each method that will allow us to iterate over the rows like so

Getting Cell Data

• Getting a String: cell.to_s('latin1')
• Getting a Float: cell.to_s('latin1')
• Getting a Int: cell.to_i
• Getting a Date: cell.date

Getting A Specific Cell

A basic script for dumping an excel file

To run the script, remember to use the -rubygems switch so that you can find the parsexcel library.

Bookmark to:

Our Task

Create a custom XML file for each .jar file in a directory.

...more after the jump!

Strategy

With any code generation project we need to identify a few things.

1. Static template: The code/text that will stay the same.
2. Dynamic variables: These are the pieces that will change in our template.
3. Input source: The input for our dynamic variables.

For me, the static template is a JNLP XML file that looks like this:

The dynamic bits will be anywhere a jar is referenced as well as the directory for the jar.

The Template

The template defines our static text, and the variables and/or ruby code that will be substituted into the text by ERB.

We use

blocks to substitute in variables from our ruby script.

Here's what my template looks like, I have four variables that I want to substitute in.

• basedir: the base directory of where I'm storing my jar.
• shortname: the name of the jar sans the version.
• jarname: the full name of the jar file.

The Code

Let's examine some code. The first thing I do is create the template string, i Use the %q{ shortcut to avoid having to worry about quotes.

Next, I need to define my template, that's what this line does.

It says, create a new ERB object using the string 'template' as the template and the "%<>" as my template escape operators.

Our next step is to loop through all the jar's in a directory and create a new file for each one, here's the juicy bits to accomplish that.

Notice that we create a binding variable called b and then pass that to our ERB object (jnlp), that binding variable tells ERB to use template variables in the same scope as it. It's some ruby black magic that makes our lives easier.
The rest of the script actually creates some directory and writes the created file to a directory based on the name of the jar.

The fun doesn't stop there

This is an elementary example, but ERB also allows you to embed real ruby code in the template, this allows you to accomplish looping.

So we could pass in an array of person objects and loop through them with a template like this:

Bookmark to:

the 2>&1 bit tells the shell to redirect stderr to stdout

Bookmark to:

Your proxy setup looks like this in your httpd.conf or vhost.conf

Then in your environment.rb add:

This will make ActionController rewrite the proper URL for your rails application.

Bookmark to:

With just a few lines of ruby we can query to our heart's content. In this post I'll go over how to create a very simple class for extracting information about movies from IMDB using the lovely Hpricot library.

Prerequisites

Downoad and Install Hpricot
That's it!

The Class

So let's think of what information we want to grab.

I think we need the title of the movie, the rating, and all the other junk - genre,plot outline, etc.

So we'll create an IMDB class that takes a IMDB url as input and can then give us all that other information.

So the beginning of our class looks like this

The Title

Now let's build our first method to grab the title. So I'll pull up my favorite movie, Lost In Translation, and take a look at the source code.

Here's the bit I'm looking for

So we need to search for a meta tag that has a name attribute of title and extract the contents of the 'content' attribute. Hpricot uses the XPath query language or a CSS selector based query language to allow us to quickly get to the data we want. Since this has a condition in it, we'll use XPath ,the first step in getting this data is determining what sort of XPath query we need to write.

Here's what we have for our title method

We use our @hp instance variable which is a Hpricot object, and tell it to return the first element with a tag of meta and an attribute that matches title, then give us the value of the content attribute.

The Ranking

Looking at the source for the rating, I see it's in a div named rating. So we'll simply search for that div, then use a regular expression to extract the number value out of it!

Here's the html

Looking closer we have the CSS query of "div.rating/b", in english that says find a bold tag in a d iv tag with the class name rating. The "/" operator we're using on @hp is a shortcut for the search method, it returns an Hprcot::Element, on the element it returns we call inner_text which returns the text of that element stripped of all html tags.

I then run the text through a regular expression to extract the rating out of 10 points.

The Rest

The rest of the main information about the movie is contained in these div's with class name info, as seen in the html here

We could extract each piece in it's own separate method, but I'm lazy so I think I'll just create one method called extrainfo that returns a hash of all the extra information back.

So our first query "div.info" is a CSS based query, it searches for all div boxes with a class of info. Now, we've got to find a way to extract the title to use as a key in the hash, and the rest of the body will be the value of the hash.

Examining the info div's closer I see the title is always stored in an h5 tag, so what we'll do is for each info div we'll grab the first h5 tag from that div. Hpricot allows you to nest queries, so the results of our first search is also searchable, and so on. So we can search each of our inf variables for all h5 tags, once we find one we simply do a little bit of regular expression magic to separate the body from the title, then store that information in the @extrainformation hash.

All Done

So our finished class now looks like this:

You could certainly get much more advanced in your IMDB scraper class, but the above should get you started. Here's a quick test script to see how our class worked.

So running this script gives us the output

More Information

Hpricot Showcase

Hpricot API Documentation

Bookmark to:

So why not the best of both worlds, MS Project and Basecamp without having to retype everything into Basecamp!

With a bit of ruby and the help of the Basecamp API we can save a lot of time.

Introduction

MS Project gives us tons of different ways to organize our project, Basecamp has only a few simple ways so our mapping can't really be one to one. Basecamp offers the following concepts: Milestones, To-Do Lists (made up of items, may be linked to a milestone).

Here is the basic strategy I used for my project plan.

MS Project Milestone = Basecamp Milestone
MS Project Tasks under Milestone = Basecamp to-do list linked to milestone.

So our script simply loops through the project, when it hits a milestone we create a milestone, we collect all the tasks into a list until we hit the next milestone and the cycle begins again. So we end up with one whole to-do list per milestone.

Prerequisites:

• Activate the Basecamp API for your project in your Account Settings (the same page where you would go to upgrade your account to pro). If you don't do this you'll see the script complain with 404 errors
• Download the ruby Basecamp API Wrapper. Put this in the same directory as your script.
• Install the xml-simple gem.

Exporting from MS Project

MS Project has a very nice export wizard. It allows you to export to a CSV file and specify filters and fields.

To access the export wizard, open up your project plan and choose File->Save As, for the 'Save as type' choose CSV (Comma delimited) (*.csv).

When prompted by the wizard to create or choose an export map, choose New map and click next, in the next dialog box choose Tasks as the type of data to export, uncheck 'Export includes headers', make sure our text delimiter is comma and click Next.

Now it's time to create our map, this is where we tell Project what columns to put in our CSV file.

For this script I just want the following data: [task name],[baseline finish date],[milestone].

So choose those three options, here's how mine looks:

Click Next, you can save the map for later if you want to do this again.

Another wrinkle, I like the following date format in MS Project: Tues 11/06/06 (Day MM/YY/DD). If you don't like it, no biggie you just have to figure out how to turn your preferred date format into YYYY-MM-DD later on in the script. I hope you like regex's!

The Script

Now it's time to write our script, so let's review what it should do:

Login to Basecamp
Select our project
Loop through our CSV
Create a collection that looks like this: Milestone->Task List
Insert milestones and task lists into basecamp!

Disclaimer: I have the ruby programming skills of a 4 year old girl, I know this. I do not claim to be an expert ruby ninja, feel free to improve the script.

Here is the script, so place this in the same directory as your csv file and the basecamp.rb file from 37 Signals.

Upload the ruby script to your hosting provider (I use dreamhost) or place in the directory on your local box.

Make sure you have your csv file from MS project and the basecamp library from 37 signals in the same directory and the xml-simple gem installed.

So, we simple change the variables for the basecamp URL, username, pass, filename and run the ruby script using our favorite editor and prepare to run the script.

For the love of god, please make sure you try this on a test account before you do it for the first time on your actual account, the results could be unexpected.

If all goes well you will see the script do it's thing like this.

If you get 404 errors, please check to make sure you have the correct URL, that you have a project created in your account, and that you have activated the basecamp API in the Account tab of your basecamp account.

This is a very simple example, depending on how complex your MS Project Plan you may want to omit the task part all together, or modify how you name tasks/milestones. Either way it sure beats entering it all by hand.

Bookmark to:

Let's say we have a table named Customers that has a firstname and lastname column but we would like to also have a fullname column available to us when doing things like iterating through Customer objects in our view. One way is to add it directly to our Customers class ( app/model/customer.rb in your rails app).

Bookmark to: