Easily Creating Fixed Length Flat Files With Groovy

Tim — Wed, 21 Jan 2009 13:42:27 +0000

Here's an (I think elegant) way to create flat files with Groovy.

I want this script to be easy to read by future developers. So if someone wants to know what the format of the file is they can see it just by opening the script.

Classes We'll Use

Groovy:

SQL: We'll use this library to access the database, it has some nice closure methods like eachRow which make iterating over results easy.
String: Groovy includes a string class that has some nice methods like padRight and padLeft, handy for making fixed length files.

Java:

File Writer: The easiest way to create files in java.

The Plan

Define our file in a map at the top of the file, it will be SQL Alias Name : Length.

Execute our SQL, for each row iterate through our file definition map and pad each value by the appropriate length.

The Code

Pretty self explanatory -- a few neat groovy things we use in this script:

Triple quotes for a multiline string (our sql)
The format method to easily format a date (bye bye simple date format) for the file name
The elvis operator ?: makes the results null safe so we can call padRight.
The each method on Groovy's Map that accepts a closure, we use that to iterate through our file definition.

import groovy.sql.Sql

import java.io.File;
import java.io.FileWriter;

def fileName = "FLATFILE" + String.format('%tY%,new Date()) + ".txt"

def sql = Sql.newInstance("jdbc:oracle:thin:@localhost:1521:midb01",
"xxx","xxx", //user,pass
"oracle.jdbc.driver.OracleDriver") //jdbc driver

FileWriter writer = new FileWriter(fileName);

//this map defines the field length based on alias id from the SQL
def fileDef = [COLUMN1:14,
COLUMN2:9,
COLUMN3:5,
COLUMN4:16]

def qry = """SELECT column1,column2,
column3,column4 FROM OurTable"""

def test = sql.eachRow(qry){row ->
fileDef.each{ k,v-> writer.append ((row[k] ?: " ").padRight(v))}
writer.append "\n"
}
writer.close()

Bookmark to:

Writing Portable Groovy Scripts with A Magic Runnable Jar

Tim — Wed, 14 Jan 2009 14:24:57 +0000

I've found myself using Java as a "scripting" language lately due to the necessity of working in mixed *nix and windows environments where I can't count on a standard scripting language being installed like python,perl, or ruby. What's more I either don't have the rights to install a binary of said scripting languages or don't want to go through the bureaucracy to request the installation.

The benefits of using Java in these environments is I can package my program as a Runnable Jar easily through Eclipse's Export functionality. I also have a wonderful wealth of libraries for doing almost anything I want -- from FTP libraries, to any Database driver I can dream of. Anyone who's had to deal with the freetds library in perl to connect to a MS SQL database knows what a huge boon being able to use JDBC is.

The cons of using Java is my code is not as terse as it could be, and for common scripting tasks like reading and writing files and database records a simple task now becomes tons of boiler plate exception handling and JDBC calls. I also have to re-compile and re-package my entire program if I want to make a simple change.

What if we could harness the power of any Java Library we wanted, no hassle deployment, but marry the simplicity of a scripting language and flexibility of an interpreted language?

Enter Groovy

Hello world in groovy:

println "Hello, I'm a portable Groovy Script!"

Iterating over an oracle record set in Groovy:

import groovy.sql.Sql

def sql = Sql.newInstance("jdbc:oracle:thin:@localhost:1521:MYSID","xxx","xxx","oracle.jdbc.driver.OracleDriver")

sql.eachRow("Select * from employees") { employee->
println employee
}

sql.close()

I don't want to go into the groovy language, that's another topic, but I think it's obvious that for simple tasks Groovy is superior to Java.

There are many ways to Run Groovy Scripts. However, I want a way to run a script without having to install the Groovy Runtime.

Ideal situation:

Develop Groovy script in Eclipse
Export Groovy Runtime as Runnable Jar from Eclipse Project
Upload Runnable Jar and Groovy Script to Any Server with a JRE
Run runnable jar server, by specifying path to groovy script

This gives me the flexibility of being able to use my favorite IDE, and easily run and change my script on any other machine with a JRE.

If you'd like you can also download the finished GroovyLauncher eclipse project.

Developing Groovy Scripts in Eclipse
You'll should have a recent version of Eclipse and the Groovy Plugin. To create a new project, simply start a new Java Project, then create a new Groovy Class to add any classes/scripts you want to your project. It's important that you add one groovy script to this project even if it's just a simple hello world script, that will prompt Eclipse to add the Groovy Libraries jars to the build path of your project.

Now to run a groovy script in eclipse you just right click on the .groovy file and do "Run as Groovy Script", done and done. However, in order to run our script from a runnable jar we are going to need a Java Class with a Main method.

Launching A Groovy Script with the Groovy Shell

The GroovyShell class is a handy Java Class that can run any Groovy Script passed into it's Main method. We're going to wrap this class in our own Main class called GroovyLauncher.

So I just make a new Java Class called GroovyLauncher and tell eclipse to create a main method for it.

The class looks like this:

import groovy.lang.GroovyShell;

public class GroovyLauncher {
public static void main(String[] args) {
GroovyShell.main(args);
}

}

So this class will be the Main Class we tell Eclipse to use when creating an exportable jar. This class does nothing inside of eclipse, because it expects the path to a groovy script to be passed into it from the command line. We'll only use this Main class when launching from our deployed location.

In order to create a Runnable Jar from an Eclipse Project we need to create a LaunchConfiguration, the quickest way to do that is just to right click on our GroovyLauncher class and do Run as Java Application. When run it will display some GroovyShell error messages because we didn't pass in any arguments. That's fine, we just want Eclilpse to create a default Launch Configuration for the class.

Now right click on your project and select "Export" in the next dialog choose Runnable Jar.

In the next dialog we need to select the Launch Configuration to use (this is the Main class that the Jar will use when run).

Pull down the drop down and you should see your GroovyLauncher Run Configuration that was created automatically by Eclipse when we tried to run our GroovyLauncher class.

Now just specify the file name for the jar and we're done. What eclipse does during the next step is repackage all the classes from your Build Path into one single jar file. So if we included mysql jdbc drivers in our build path, they'd be repackaged into our own jar. This is a legal gray area, but it's damn cool and simple. We don't have to worry about classpaths anymore when we launch our jar! We just run it with "java -jar filename.jar"

So now to test our GroovyLauncher jar we can run it on our machine or another by running:

java -jar GroovyLauncher.jar ourscript.groovy

Now we can pass in ANY groovy script we want using this same launcher jar, it can reference any Java class that was in our BuildPath of our eclipse project. If you didn't add anything other than the Groovy Libraries you'll just have access to the core Groovy Libraries + the Java API's, if you want to be able to do SQL and other fun stuff you'll need to add the appropriate jars to your Eclipse Project just like normal.

Conclusion
You might make a couple different Runnable Jars based on different needs. You could create a project that just included the Groovy Libraries and the GroovyLauncher class and maybe export that as GroovyLauncherLight.jar and make another project that has your favorite JDBC drivers plus any other common libraries and export that project as GroovyLauncherFull.jar

A word of warning, I haven't benchmarked using the GroovyShell versus the GroovyLauncher, however most of your swiss army type scripts don't need to be blazing fast, they just need to be easily changed and easy to run.