Memcached in Grails: a quickie

Grails, programming

Two clients will be covered: XMemcached and spymemcached

Xmemcached

Add dependency

Add in BuildConfig.groovy:

dependencies {
...
compile 'com.googlecode.xmemcached:xmemcached:1.4.2'
}

Set up the bean

Add in conf/spring/resources.groovy:

beans = {
...
memcachedClient(net.rubyeye.xmemcached.utils.XMemcachedClientFactoryBean) {
servers="localhost:11211"
}
}

Usage in a service

class MyService {
static final int expiry = 30 // 30s

def memcachedClient

def perform() {
def val = memcachedClient.get('key')
if (!val) {
// cache miss
...
// put in cache
memcachedClient.set('key', expiry, val)
}
// use val
}
...
}

Spy Memcached

Add dependency

Add in BuildConfig.groovy:

dependencies {
...
compile 'net.spy:spymemcached:2.9.1'
}

Set up the bean

Add in conf/spring/resources.groovy:

beans = {
...
memcachedClient(net.spy.memcached.spring.MemcachedClientFactoryBean) {
servers="localhost:11211"
}
}

Usage in a service


class MyService {
static final int expiry = 30 // 30s

def memcachedClient

def perform() {
def val = memcachedClient.get('key')
if (!val) {
// cache miss
...
// put in cache
memcachedClient.set('key', expiry, val)
}
// use val
}
...
}

References

Notes on multiple Grails apps sharing a DB

Grails, programming

Multiple Apps

Two logical partitions of services, perhaps one public-facing with GSPs serving up UI and one headless with REST services, can be easier to maintain if implemented as separate Grails apps.

If there is a breaking change in a domain (and corresponding tables) in one app, for instance, it can be re-deployed without disrupting the other app as long as that other app doesn’t use the domain in question.

Shared domains in in-place plugins

On the other hand, there may be domains that are relevant and used by both apps. One way to make this work is to put them in a plug-in, referenced by both apps. This plug-in can even be an in-place plug-in if the iteration of editing and publishing the plug-in proves tedious during development.

The trouble with in-place plugins

Various tools support an evolving in-place plug-in with varying success (Groovy/Grails Tool Suite 3.3.0, for instance, doesn’t even seem to recognize them). IntelliJ IDEA 12.1 Ultimate (the paid version) seems to recognize them and handles them fine most of the time … because even Grails itself seems flaky at times, requiring periodic cleaning up of cached binaries of apps that use in-place plugins:

rm -rf ~/.grails/.slcache
rm -rf ~/.grails/2.2.3/projects/myApp

database-migration

After about a week of Grails development, most people will realize that they’ve outgrown that dataSource’s dbCreate="create" option. And for a while after that point, perhaps dbCreate="update" may buy some more time. Beyond that, some real SQL migration tool is needed.

Rails has “rake db:migrate“, and Grails has “grails dbm-update” with the database-migration plugin (currently 1.3.5).

Unfortunately, while the database-migration plugin works great for a single Grails app connecting to an SQL DB, it is very cumbersome to get working for multiple apps sharing an SQL DB.

Disjoint domains

If app1 has domain A and app2 has domain B, and the shared DB naturally has tables a and b, running dbm-gorm-diff in app1 will give you a changeset that will attempt to drop table b. Similarly, running dbm-gorm-diff in app2 will give you a changeset that will attempt to drop table a. This is because app1 doesn’t know about domain B and therefore table b. The dbm-gorm-diff tool will assume that domain B has been deleted, so it dutifully adds an entry to clean up table b for you. And similarly, running dbm-gorm-diff in app2 will add an entry to drop table a.

One way I found to get around this is to add app2’s tables (including joining tables and sequences thereof) to app1’s grails.plugin.databasemigration.ignoredObjects and app1’s tables and friends into app2’s grails.plugin.databasemigration.ignoredObjects.

But that’s pretty tedious work and a maintenance pain.

Shared domains

Then there are the placements of these changelog files. The dbm-* tools look in only one place for them: grails-app/migration. So that means there are N lists of these files if I have N apps sharing a DB. So in the least there is a duplicate of the base/root changelog file in each app’s grails-app/migrations/. Then each app has its own changelog files for changes to its own domains.

What about shared domains? Those from the in-place plugin? Where do changelogs for those domains go? To avoid problems, of course, they need to go into all the apps (not strictly necessary, but for consistency and documentation, better to replicate these), just like the base/root changelog.

So MORE maintenance steps.

At this point one wonders if it’s better to craft these changelogs manually. But for the most part, dbm-gorm-diff saves a lot of work. So I’m not entirely ready to let it go yet.

Policy

I presented this to my team, and they agreed that maintaining separate changelog files and ignored lists are tedious and error-prone. We ended up biting the bullet and take the one-schema strategy:

  • All domains in general should reside in the shared in-place plug-in, whether they were actually shared between apps or not
  • Pick one of the apps from which to perform database-migration tasks, and put the changelogs in that app

The result is that each app gets the union of all the domains, and each of the apps use a proper subset of that group of domains. It’s not as lean and pure, but it’s a good compromise, I believe.

So when there is a schema change, we now need to assess the impacted apps and redeploy each as needed.

Grails unit vs integration tests

Grails, programming, test

Services

Content here applies to Grails 2.1.3 and may also apply to newer versions.

Setup

We start with three service classes: OneService, TwoService, and ThreeService. OneService uses TwoService which uses ThreeService.

class ThreeService {
def hello() {
"Hello from Three"
}
}

class TwoService {
def threeService
def hello() {
"Hello from Two:" + threeService.hello()
}
}

class OneService {
def twoService
def hello() {
"Hello from One:" + twoService.hello()
}
}

Calling OneService#hello() would normally return "Hello from One:Hello from Two:Hello from Three"

Unit Test

Add a unit test for OneService:

@TestFor(OneService)
class OneServiceTests {
void testSomething() {
defineBeans {
twoService(TwoService)
threeService(ThreeService)
}
service.hello()
}
}

Run it and get this:

| Failure: testSomething(org.van.OneServiceTests)
| java.lang.NullPointerException: Cannot invoke method hello() on null object
at org.van.TwoService.hello(TwoService.groovy:7)
at org.van.OneService.hello(OneService.groovy:7)

The defineBeans{ ... } trick only works for one level, it seems. It got AService#bService hooked up, but BService#cService remained null.

So what happens is that OneService#hello() got to call TwoService#hello() OK. However, when TwoService#hello() tries to call ThreeService#hello(), we get a NPE because TwoService#threeService was not properly injected.

Integration Test

One way to test the flow is to use an integration test instead:

class OneServiceIntTestTests {

@Test
void testSomething() {
OneService oneService = new OneService()
TwoService twoService = new TwoService()
ThreeService threeService = new ThreeService()
twoService.threeService = threeService
oneService.twoService = twoService
oneService.hello()
}
}

While this works, we will need to do the injection plumbing ourselves.

Controllers

Setup

Add a TwoController class:

class TwoController {
def twoService

def index() {
render twoService.hello()
}
}

Accessing the controller returns Hello from Two:Hello from Three as expected.

Unit Test

Add a unit test for TwoController:

@TestFor(TwoController)
class TwoControllerTests {
void testSomething() {
defineBeans {
twoService(TwoService)
threeService(ThreeService)
}
controller.index()
}
}

And it fails similarly to OneServiceTests:

| Failure: testSomething(org.van.TwoControllerTests)
| java.lang.NullPointerException: Cannot invoke method hello() on null object
at org.van.TwoService.hello(TwoService.groovy:7)
at org.van.TwoController.index(TwoController.groovy:7)

Again, the defineBeans { ... } mechanism resolved TwoController#twoService. But once we call into TwoService#hello, it throws an NPE because TwoService#threeService is not properly injected.

Integration Test

And here’s the integration test for TwoController:

class TwoControllerIntTestTests {
@Test
void testSomething() {
TwoController twoController = new TwoController()
TwoService twoService = new TwoService()
ThreeService threeService = new ThreeService()
twoService.threeService = threeService
twoController.twoService = twoService
twoController.index()

assertEquals "Expected output", 'Hello from Two:Hello from Three', twoController.response.text
}
}

Summary

Unit tests, with their defineBeans { ... } mechanism, only helps inject one level of beans for the target (i.e. @TestFor()) class. If the injected beans used by that class then have more injected beans of their own, those are not injected correctly.

This works identically for services and controllers.

Integration tests allow for deeper chaining of injected beans. However, the construction/initialization and the actual injection have to be done manually.

Reference

Grails testing guide

SampleServlet: Java web app starter

Java, jetty, maven, programming

Prerequisite

Maven 3.x installed – try http://maven.apache.org/download.cgi
Important: earlier versions of Maven, such as 2.x, won’t work correctly

Set-up

Locate a subdirectory and run:

mvn archetype:generate

When prompted, use maven-archetype-quickstart as the archetype to use.  Enter the groupId, artifactId, version, etc.

Add dependencies and plugins into pom.xml

A relatively useless pom.xml file is generated.  Modify it to include some dependencies and plugins:

<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>org.van</groupId>
<artifactId>sampleweb</artifactId>
<packaging>jar</packaging>
<version>1.0</version>
<name>sampleweb</name>
<url>http://maven.apache.org</url>
<dependencies>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>3.8.1</version>
<scope>test</scope>
</dependency>

<dependency>
<groupId>javax.servlet</groupId>
<artifactId>servlet-api</artifactId>
<version>2.5</version>
</dependency>
</dependencies>

<build>
<finalName>sampleweb</finalName>
<plugins>

<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>2.0.2</version>
<configuration>
<source>1.6</source>
<target>1.6</target>
</configuration>
</plugin>
<plugin>
<groupId>org.mortbay.jetty</groupId>
<artifactId>jetty-maven-plugin</artifactId>
<configuration>
<contextPath>/</contextPath>
</configuration>
</plugin>

</plugins>
</build>
</project>

Add web.xml per Java Servlet specs

Create the subdirectory src/main/webapp/WEB-INF/ and add into it <strong?web.xml:

<?xml version="1.0" encoding="UTF-8"?>
<web-app xmlns="http://java.sun.com/xml/ns/javaee"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-app_2_5.xsd"
version="2.5">
<servlet>
<servlet-name>SampleServlet</servlet-name>
<servlet-class>org.van.SampleServlet</servlet-class>
</servlet>

<servlet-mapping>
<servlet-name>SampleServlet</servlet-name>
<url-pattern>/SampleServlet/*</url-pattern>
</servlet-mapping>
</web-app>

Create SampleServlet.java

Under src/main/java/, delete the App.java generated and add the servlet file under the appropriate package subdirectory (e.g. “org/van/SampleServlet.java”):

package org.van;

import javax.servlet.ServletException;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

public class SampleServlet extends HttpServlet {
/*
Redirects to some other page. Sample usage:
http://localhost:8080/SampleServlet?destination=http://www.therealvan.com
*/
public void doGet(HttpServletRequest req, HttpServletResponse resp)
throws ServletException, java.io.IOException {

String dest = req.getParameter("destination");
resp.sendRedirect(dest);
}
}

Run

Just invoke the Jetty run command:

mvn jetty:run

Zip of starter project

Here is a zip of the starter project to extract wherever to get started:
sampleweb.zip

Details

The sample project uses the following:

  • Java Servlet 2.5 API
  • Maven compiler plugin
  • jetty-maven-plugin

Grails run-script with parameters

Grails, programming

Scripts vs. parameters

The “grails run-script” task allows running a bunch of scripts, and therein lies the problem: the parameters passed to the command are interpreted by the command to be names of scripts to run:

grails run-script ~/script1.groovy ~/script2.groovy ~/script3.groovy

This works fine if the scripts are self-contained and did not expect any parameters on their own.  What if ~/script1.groovy expects some parameters?

The problem is that the RunScript.groovy script assumes each parameter to be script files to run, so there is no support for parameters to any of the script files run.

So I would like to trade:

  • the ability to run multiple scripts with one call to run-script

with

  • multiple calls to something like run-script-with-params

I figured that, if I were to be calling scripts in a batch, making multiple calls shouldn’t be a big deal. Not being able to pass parameters to individual scripts, on the other hand, IS a big deal.

A tweak of the script RunScript.groovy (below) can help with that:

UPDATE: I just found that a similar script (even the name is remarkably similar) has already been suggested here.  (Scroll down the comment section to see the suggestion.)  Oh well.  Wish it was added to Grails so I didn’t have to spend so much time on this.

import org.springframework.orm.hibernate3.SessionFactoryUtils
import org.springframework.orm.hibernate3.SessionHolder
import org.springframework.transaction.support.TransactionSynchronizationManager

/**
* GANT script to run an arbitrary Groovy script with GORM and Spring context and also with parameters
*
* A sample invocation of a script:
* grails run-script-with-params ~/script1.groovy -enableLog -name=van
*
* Arguments are processed as follows:
* --xx=yy or -xx=yy are parsed into key/value pairs added into argsMap and passed to the script
* --xx or -xx are interpreted as xx:true and treated as a key/value pair and added into argsMap
* xxxxx are treated as a parameter and stored into the array argsMap.params
*
* The FIRST parameter is assumed to be the path to the script file to run.
* The rest of the parameters will be parsed into a map passed onto the script as argsMap
*
* This script is based on and inspired by
* {@link RunScript}
*/
includeTargets << grailsScript("_GrailsBootstrap")

target(main: "GANT script to run an arbitrary Groovy script with GORM and Spring context") {
depends checkVersion, configureProxy, bootstrap, runScript

}

target(runScript: 'Runs a Groovy script file with the parameters propagated to it') {
configureHibernateSession()

if (!argsMap.params) {
event('StatusError', ['ERROR: Required script name parameter is missing'])
System.exit 1
}
executeScriptWithArgsMap argsMap.params[0], classLoader, argsMap
}

def executeScriptWithArgsMap(scriptFile, classLoader, argsMap) {
File script = new File(scriptFile)
if (!script.exists()) {
event('StatusError', ["ERROR: Script $scriptFile does not exist"])
System.exit 1
}
event('StatusUpdate', ["Running script $scriptFile ..."])
def persistenceInterceptor = appCtx.containsBean('persistenceInterceptor') ? appCtx.persistenceInterceptor : null
persistenceInterceptor?.init()
try {
def binding = new Binding(ctx:appCtx, grailsApplication:grailsApp, argsMap: argsMap)
def shell = new GroovyShell(classLoader, binding)
shell.evaluate script.text
event('StatusUpdate', ["Script $scriptFile complete!"])
} finally {
persistenceInterceptor?.flush()
persistenceInterceptor?.destroy()
}
}

def configureHibernateSession() {
// bind a Hibernate Session to avoid lazy initialization exceptions
TransactionSynchronizationManager.bindResource(appCtx.sessionFactory,
new SessionHolder(SessionFactoryUtils.getSession(appCtx.sessionFactory, true)))
}

setDefaultTarget(main)

The trick is how we start a Groovy shell, passing it a Binding object:

def binding = new Binding(ctx:appCtx, grailsApplication:grailsApp, argsMap: argsMap)
def shell = new GroovyShell(classLoader, binding)

We added argsMap to the Binding object passed to the shell. And this allows the target script to now have access to argsMap, in addition to the already available ctx (the Spring application context) and grailsApplication (the GrailsApplication object).

Put the script above into the Grails project’s script subdir as /scripts/RunScriptWithParams.groovy and we can now run this from the Grails subdir:

grails run-script-with-params ~/script1.groovy --enableThis --param1=val1

The target script (in this case ~/script1.groovy) will be able to get the parameters via the argsMap global variable injected into the shell:

println "enableThis = ${argsMap.enableThis}, param1 = ${argsMap.param1}"

Windows caveat

The only caveat I found is that, when running in the Windows command prompt, parameters with a format --key=value need to be in quotes:

grails run-script-with-params ~/script1.groovy --enableThis "--param1=val1"

Running under Cygwin’s terminal works fine without the quotes, however.  So it’s something special about the Windows command.exe or cmd.exe.

Grails class names

Grails, programming

Seems like there are many flavors of the name for a Grails class:

  • fullName — Returns the full name of the class in the application with the the trailing convention part and with the package name
  • name — Returns the logical name of the class in the application without the trailing convention part if applicable and without the package name.
  • naturalName — Returns the name of the property in natural terms (eg. ‘lastName’ becomes ‘Last Name’)
  • shortName — Returns the short name of the class without package prefix
  • propertyName — Returns the name of the class as a property name
  • logicalPropertyName — Returns the logical name of the class as a property name

These are from http://grails.org/doc/2.2.x/api/org/codehaus/groovy/grails/commons/GrailsClass.html.  It looks like somebody just typed in something just to have something show up in Javadoc.

A pause in the code to show an example is probably the best way to document these names.  For a class like com.blah.UserManagementController:

  • fullName: com.blah.UserManagementController
  • name: UserManagement
  • naturalName: User Management Controller
  • shortName: UserManagementController
  • propertyName: userManagementController
  • logicalPropertyName: userManagement

Now that that’s out of the way, notes:

Spring’s application context (e.g. grailsApplication.mainContext) holds on to its beans using names that may not be obvious. The easiest way to figure out is to fire up

grails console

and evaluate

grailsApplication.mainContext.getBeanDefinitionNames()

and find the bean’s name.

Grails scripting

Grails, programming

Within any Web application, there is the Web site, which is pretty well served by Grails.  Very soon, there will inevitably be some background processing that needs to be done for the holistic system.  These background tasks are typically:

  • background in nature such that typical users won’t know of their existence
  • process-intensive with little or no interactive flows
  • console stdin/stdout for scripting
  • independent of the Web app’s life cycle / process space

With respect to Grails, the Web app has GORM and all its convenient properties abstracted in very concise domain classes.  However, these are only available to the Grails run-time.  There are posts (this and the like) that talks about having multiple Grails apps sharing the domain objects, but that doesn’t fit the profiles listed above since I’d have two interactive Web apps, albeit one for public and one for admin/internal controllers and services.

An alternative is to look at Grails scripting, of which there are two flavors that I know of:

  • Grails scripts
  • Custom Groovy scripts

Both of these seem to be able to get at the Spring ApplicationContext and transitively all its beans.

Grails scripts (create-script)

Create these using:

grails create-script MyScript

A script template will be added under the /scripts subdir (e.g. /scripts/MyScript.groovy):

includeTargets << grailsScript("_GrailsBootstrap")

target(main: "The description of the script goes here!") {
    // ...

}

setDefaultTarget(main)

Within the body of target(...) { }, we can add a list of dependencies and then use “appCtx” to refer to the Spring ApplicationContext:

target(...) {
    depends(configureProxy, packageApp, classpath, loadApp, configureApp, compile)

    def myService = appCtx.myService
    println(myService.doSomething())
}

Note the use of appCtx to reference the Spring ApplicationContext.

The script can be moved to these places also:

  • $GRAILS_HOME/scripts
  • $USER_HOME/.grails/scripts
  • $PROJECT_HOME/plugins/*/scripts

To run it:

cd $PROJECT_HOME
grails MyScript

Note the conciseness of the command.

Custom Groovy scripts (run-scripts)

This approach allows writing less coupled Groovy scripts that can be placed anywhere on the file system (not confined under the project’s /scripts subdir, for instance).

The price to pay for that flexibility are:

  • The working directory when I run the script still must be where the Grails project is
  • Getting the reference to the ApplicationContext is quite convoluted a process

So I can create a script at ~/scripts/grails/myapp/MyService.groovy:

def appCtx = new User().domainClass.grailsApplication.mainContext
def myService = appCtx.myService

println(myService.doSomething())

Note how I got the appCtx instance: creating a bean (must be properly imported or qualified) class, then walk the reference to the domainClass and then grailsApplication and then mainContext.

To run it:

cd $PROJECT_HOME
grails run-script ~/scripts/grails/myapp/MyScript.groovy

Note the use of the run-script target and the full path to the script file. Since the script file can be anywhere, there can certainly be many versions of the script file.

Similarities and Differences

Both types of scripts have full access to GORM and Spring.

Scripts created by “grails create-script”:

can only reside in one of four locations

  • $GRAILS_HOME/scripts
  • $USER_HOME/.grails/scripts
  • $PROJECT_HOME/scripts
  • $PROJECT_HOME/plugins/*/scripts

can be invoked simply via “grails <script name>” (the .groovy suffix is not required)

can reuse some of the _Grails* scripts that came with Grails (e.g. _GrailsBootstrap)

does NOT have access to $PROJECT_HOME/src/groovy or $PROJECT_HOME/src/java during run-time

Scripts run by “grails run-script”:

can reside outside $PROJECT_HOME.  However, when running, the working subdirectory needs to be $PROJECT_HOME

requires full path to *.groovy file when run using “grails run-script <full path to script file>.groovy”

HAS access to $PROJECT_HOME/src/groovy or $PROJECT_HOME/src/java during run-time

Reference

Page on create-script from Grails

Page on run-script from Grails