Spring @Configurable With AJDT Compile-Time Weaving

I know, that’s a honey of a title, but you found this didn’t you?

Anyways, after some house cleaning and re-organizing of the projects that make up RightsPro in Eclipse I found that domain objects annotated with Spring’s @Configurable annotation were no longer having their dependencies injected when running the app in tomcat or when running AbstractTransactionalDataSourceSpringContextTests unit tests.

Turns out I must have removed the AspectJ library during re-organization and added it back in which also seemed to wipe out the AspectJ Build settings in the project properties.  When AspectJ does its compile-time weaving it doesn’t know anything about the @Configurable annotation unless you add spring-aspects.jar to the Aspect Path tab in the AspectJ Build settings.

F yeah, magical aspects are back in business.

Render Me This: Video Thumbnails in Alfresco 3.3

Alfresco has undergone quite a few changes under the hood in version 3.3, including refactoring of the ThumbnailService to make use of the RenditionService, which I had to explore in a recent deep dive to get video working for the thumbnails extension. This is obviously a subject dear to your heart as well or you would have moved on by now.

Alfresco 3.1.

Let’s take a look at how things worked back in the old days, before 3.2 (fuzzy 2009 dream sequence begins…)

You most likely kicked things off with a thumbnail generating URL which would hit a ScriptNode.createThumbnail method which looked up the thumbnailDefinition and called the ThumbnailService with the TransformationOptions contained in the definition.

The ThumbnailService then told the ContentService to perform the transformation which looked up the ContentTransformer from the TransformerRegistry and executed transformer.transform which usually delegated to a RuntimeExec object to perform the actual command line transformation.

If we were to visualize the key elements of that stack for a synchronous video thumbnail creation it would look something like:

  • org.alfresco.repo.jscript.ScriptNode.createThumbnail
    • org.alfresco.repo.thumbnail.ThumbnailRegistry.getThumbnailDefinition (loads TransformationOptions)
    • org.alfresco.repo.thumbnail.ThumbnailServiceImpl.createThumbnail
      • org.alfresco.repo.content.ContentServiceImpl.transform
        • org.alfresco.repo.content.transform.ContentTransformerRegistry.getTransformer
        • org.alfresco.repo.content.transform.ContentTransformer.transform
          • TempFileProvider.createTempFile
          • org.alfresco.util.exec.RuntimeExec.execute
            • ffmpeg …

Well now, that’s not so bad.

Alfresco 3.3.

These Alfresco people don’t mess around. If it’s determined that refactoring should happen, it happens, real damn quick, even if it’s huge. To that end subsystems were rolled out in 3.2 and the RenditionService was introduced in 3.3.

Subsystems. Certain content transformers, like ImageMagick, have been refactored as a subsystem, which is cool. Read the wiki page to get the full story but in short it’s a much more flexible architecture.

The ffmpeg video transformer contained in the latest release of the thumbnails extension follows this precedence and has been refactored as a subsystem, so there.

RenditionService. I wouldn’t say I hate it, but this package should be wary of eating crackers in bed, or maybe I’m just not familiar enough with it yet.

The new rendition service utilizes RenditionDefinitions which require that any TransformationOptions be ‘serialized’ into a parameter map, which are then reassembled into TransformationOptions objects before being passed to the ContentTransformer. I suppose this is done to make it easier to do things like hand renditions off to another server in a cluster, but it’s a bit of a pain in the ass for developers.

So, back to our synchronous video thumbnail creation scenario (deep breath), the ThumbnailService now creates a RenditionDefinition which it passes to the RenditionService which wraps things up in a PerformRenditionActionExecuter that gets passed to the ActionService. The PerformRenditionActionExecuter calls the ActionService again which loads the RenderingEngine from the spring ApplicationContext. A RenderingEngine is itself an ActionExecuter so the ActionService calls execute which calls render which proceeds to rebuild the TransformationOptions object needed by the ContentService to get the proper ContentTransformer so transformer.transform can use a RuntimeExec object to perform the actual command line transformation.

The key elements of our pseudo stack/method trace would look like:

  • org.alfresco.repo.jscript.ScriptNode.createThumbnail
    • org.alfresco.repo.thumbnail.ThumbnailRegistry.getThumbnailDefinition (loads TransformationOptions)
    • org.alfresco.repo.thumbnail.ThumbnailServiceImpl.createThumbnail
      • org.alfresco.repo.thumbnail.ThumbnailServiceImpl.createRenditionDefinition
        • org.alfresco.repo.thumbnail.ThumbnailRegistry.getThumbnailRenditionConvertor().convert(transformationOptions, assocDetails) <- this serializes the transformationOptions into a parameter map
      • org.alfresco.repo.rendition.RenditionServiceImpl.render
        • org.alfresco.repo.rendition.RenditionServiceImpl.createAndExecuteRenditionAction
          • org.alfresco.repo.action.ActionServiceImpl.createAction(PerformRenditionActionExecuter.NAME)
          • set the renditionDefintion on performRenditionAction
          • org.alfresco.repo.action.ActionServiceImpl.executeAction
            • org.alfresco.repo.action.ActionServiceImpl.directActionExecution
              • load ActionExecuter from applicationContext
              • org.alfresco.repo.rendition.executer.ImageRenderingEngine.execute
                • org.alfresco.repo.rendition.executer.ImageRenderingEngine.render
                  • org.alfresco.repo.rendition.executer.ImageRenderingEngine.getTransformationOptions <- rebuilds parameter map into TransformationOptions objects
                  • org.alfresco.repo.content.ContentServiceImpl.transform
                    • org.alfresco.repo.content.transform.ContentTransformerRegistry.getTransformer
                    • org.alfresco.repo.content.transform.ProxyContentTransformer.transform <- subsystems
                      • org.alfresco.repo.content.transform.ContentTransformerWorker.transform
                        • TempFileProvider.createTempFile
                        • org.alfresco.util.exec.RuntimeExec.execute
                          • ffmpeg …

that’s a spicy meatball!

A while ago we submitted an issue to Alfresco suggesting TransformationOptions which contain distinct target AND source options so that one could for example specify that all ‘doclib’ thumbnails be of max-width X and max-height Y (the type of target options currently available) and furthermore, if it’s a document take page 1 (or maybe you need a different page in your deployment) or if it’s a video take 1 frame starting 0.5 seconds in, or if it’s a layered image file take the second layer, etc. (a list of source options).

Alas, this concept of SourceTargetTransformationOptions hasn’t yet been embraced by the Alfresco team but is used by the video transformer in the thumbnails extension which made refactoring for renditions even more difficult, but you’re not here to listen to me bitch and moan, so I’ll just say that it’s done and seems to be working fine.

So, there you have it, renditions and subsystems as they relate to video thumbnails. Hit me up with any questions or things I’ve missed.

Daemontools and File Permissions

Contegix turned me on to daemontools which is a great way to manage services you need to keep alive.  A great use for it is managing multiple Apache Tomcat instances which can each be easily be configured to run under different users with different options.

I was recently running into a problem where a webapp running in Tomcat didn’t have access to a particular directory mounted via fuse.  I could verify that the user Tomcat was running under had access to the directory by logging in as that user then creating a file there.  WTF?

Turns out the run script being used for the Tomcat service was using the daemontools program setuidgid, i.e.

exec setuidgid ${USER+"$USER"} ./bin/catalina.sh run

and the daemontools manual for that program states that it removes “all supplementary groups“.  Doh!

The Tomcat user’s access to this particular directory is in fact granted based on a secondary group membership and in this case should really stay that way.  setuidgid only allows a single user argument that’s used for both the user and group so I couldn’t change the running group and there’s no option to keep the user’s secondary groups.

Instead I opted to use su instead of setuidgid which leaves the secondary groups intact and gave the desired access to the directory:

exec su ${USER+"$USER"} -c "./bin/catalina.sh run"

Run catalina, run.

Alfresco Video Thumbnails

It seems like every 6 months to never I hear: “The Alfresco thumbnails forge project used to be cool man, what happened?”  Chill Winston, it’s still cool.

Now that Alfresco has a native implementation of thumbnails is the forge project still needed?  Well, ‘needed’ is relative I suppose, but check out what the project gets you beyond the basic Alfresco distribution:

  • Thumbnails when browsing, searching, or viewing a document in Alfresco Explorer (i.e. not Share)
  • CoolIris view when browsing or searching in Alfresco Explorer (which is sweet)
  • Actions to force the creation or update of thumbnails using Alfresco’s native thumbnail service
  • A video transformer based on ffmpeg with the ability to specify an offset (also sweet)
  • A patch to delete and regenerate thumbnails created by old versions of the forge project (Admittedly, not that awesome, but useful nonetheless.  No one wants a messy bloated repo.)

There’s also several goodies on the roadmap for future releases of the project including Share mods.

So what are you waiting for? Go manage your modules and pop that amp into your war (for the non-geeks, I promise that’s not gross).

Alfresco 3 Thumbnails and the Forge Project

The thumbnails forge project was started quite a while ago (like Alfresco 0.9 or so… old school) and the first public release was put on the forge in late 2006 (which would have been around Alfresco 1.3 I guess).

The project gained the attention of the Alfresco team and they contacted me about making a few changes to bring it more inline with what they had in mind for an implementation, notably creating a public thumbnail service.  Those changes were made and I continued to collaborate with Alfresco on the forge project and gave input on their implementation.

Alfresco 3 now has a native implementation of thumbnails and version 1.0 of the forge project migrated to the new architecture, but it’s still cool.

Here’s a look at some of the differences to be aware of between the old thumbnails project implementation and Alfresco’s new native thumbnails:

Concept Pre 1.0 Versions of Forge Project Alfresco 3 Native
The service responsible for generating and retrieving thumbnails
ThumbnailService

The generateThumbnails method would create thumbnails for
all ThumbnailSpecifications defined.

A specific thumbnail can be generated with overriding options
(possibly defined by a user in the web interface) which
are merged with the defaults of the
ThumbnailSpecification
(useful in defining the offset for a video thumbnail for example).

ThumbnailService

The createThumbnail and updateThumbnail
methods are given the specific TransformationOptions to
use in generating the thumbnail.

The content property from which the source data will be read
must be specified when generating the thumbnail, the default cm:content
will not be assumed.

A common size and destination mimetype for thumbnails
ThumbnailSpecification

ContentTransformers and corresponging command line options explicitly
defined in config

ThumbnailDefinition

ContentTransformer determined by ContentService
from the given TransformationOptions

The detailed resizing specifications
TransformerSpecification

The specific ContentTransformer to use and the necessary
command line options are defined

TransformationOptions

The resizing details are specified using classes containing generic
definitions (for example ImageResizeOptions used by
the ImageTransformationOptions class) which are then
translated to appropriate command line code by the ContentTransformer
found by the ContentService

The model type
tn:thumbnail

<parent>cm:cmobject</parent>

Not being of type content prevents most rules from being run on
thumbnail nodes lest we do something unnecessary and resource intensive
like extract metadata or even create another thumbnail of a thumbnail node
and prevents thumbnail nodes from being indexed or returned in a search.

cm:thumbnail

<parent>cm:content</parent>

Care must be taken to not run rules or index thumbnail nodes since they
extend content.

One suggestion has been to add the following to the thumbnail type:

<aspect name="rule:ignoreInheritedRules">

<title>Ignore Inherited Rules</title>

</aspect>

The model property name of the common size and mimetype destination definition
tn:specificationName

The specifiationName was the name of the
ThumbnailSpecification used to generate the thumbnail.

cm:thumbnailName

The new CreateThumbnailActionExecuter uses the user-chosen
thumbnailName to retrieve the
ThumbnailDefinition but the relationship between the
ThumbnailDefinition and its thumbnailName is
not enforced by the ThumbnailService.

Default thumbnail sizes (max length/width)
SMALL 80px JPEG
MEDIUM 160px JPEG
medium 100px JPEG
doclib 100px PNG
webpreview SWF
imgpreview 480px PNG
avatar 64px PNG
  1. Strikethroughs represent deprecated items.
  2. Avatar thumbnail size has nothing to do with what I can only assume is an incredibly overhyped movie about pissed off blue tree people since I haven’t been to a theater in like 10 years.

This information has been available in the thumbnails project’s 1.0 release readme but as it’s not shown by default I figured I’d post here as well.

Jungle Disk Command Line 86ed in 3.0

[picapp align=”right” wrap=”true” link=”term=thumbs+down&iid=5214086″ src=”5/d/1/f/Crowd_Looking_up_4343.jpg?adImageId=9910668&imageId=5214086″ width=”234″ height=”299″ /]My favorite backup program/service, Jungle Disk, is a cross-platform app which uses AES-256 encryption and stores files on either Amazon S3 or the Rackspace cloud (though I currently can’t find any reference to the Rackspace option on their site).

Unfortunately they have ripped out one of the main reasons I turned to the solution in the first place, a simple command line interface in Linux, a decision aggravating many of their users.

I find it hard to believe the original developer would have supported a decision like this, but alas, Jungle Dave has left the building.

Rackspace, the company who purchased Jungle Disk, does seem to be willing to add the feature back into its most expensive Server Edition of the product, but hasn’t yet shown much interest in reinstating it in the other versions.

The latest release that still supports the CLI can be found here and the readme in the release guides one through setting up a mount point via FUSE.

Let’s hope Rackspace adds the CLI back into Jungle Disk 3.0 soon so we can all get our headless back up back on.

Update 20100216: According to responses in the forum thread it seems the Jungle Disk team will be returning the CLI to the product in the next release, and there was much rejoicing.

Update 20100314: On 20100305 Jungle Disk announced 3.05 was available.  I’ve finally gotten around to playing with it and it seems to work as expected.  Thanks JD.

GApps Mail and Calendar Problems

My RightsPro Google Apps for business mail account will occasionally stop receiving mail to sent to groups I’m a member of, a huge pain in the ass indeed.

These messages can be found in Postini’s archive service and are marked as delivered normally but there’s no sign of them in the GApps mail account’s spam or inbox. The sender does not get a bounce message and there’s no way that I know I’m not receiving them, save searching Postini’s archive for mail sent to every group I’m a member of.

We’re GApps premium customers so I of course contacted GApps support on the issue. The normal progression of a GApps support ticket is just terrible: little to no activity and unhelpful or downright nonsensical responses when you do get one.  That is until you get the ticket escalated at which point their ‘specialists’ are actually quite helpful and willing to spend as long as it takes with you, even on the phone for an hour or more, to resolve the issue as best they can (many thanks John in Dublin).

GApps support called my issue a ‘hard bouncing state‘. This state was cleared and and my group emails flowed normally for about a month, then the same behavior returned. Apparently this state is associated with the user, not the group membership or group. So the group doesn’t go into a bouncing state (other members receive emails to the group normally), and the member within the group doesn’t go into a bouncing state, but the user itself goes into a bouncing state for all groups. Emails addressed to nicknames still go through.

The emails that caused this bouncing state were spam with a spoofed sender. <rant>I won’t go into the details of how much I loath spammers but suffice it to say that they are completely worthless human beings that leech off of society and should probably be shot on sight</rant>. GApps support couldn’t immediately retrieve the sender address but my guess is that they’re sent from a group which I’m the only member of, since no one else in our domain has this problem.

It doesn’t look like there’s any way for me to clear this bounced state myself but support is looking into that and exactly how and why the bounce state is caused in the first place.

I had what seems to be a related issue where invites from Google calendars were missing for that same account. Somehow the notification settings for my default calendar were all turned off (I never changed them, really) but after clearing this bounce state and turning the notifications on the invites show up. You may want to double check that your GApps for business notifications are not disabled by default:

Edit the main calendar notifications
Enable the notifications you want

I’ll update this post as I find out more about this hard bouncing state, how to avoid it, or how to clear it.

Update 20100204: GApps support has recommended a couple of thing to try in Postini:

  1. Put IP locking in place such messages with a from address in our domain would only be accepted if coming from the GApps cloud or one of our servers.
  2. Turn off blatant spam bouncing.

The former seems like a pain to maintain current IPs and I’m sure someone would forget about it at some point and be frantically looking for the cause of why some messages were bouncing until we realized it was our own damn fault and just wasted several precious hours that could have been spent with the family, or a beer, or both, so we implemented option 2.

Log in to the System Administration side of Postini…

After selecting your org click Spam Filtering
Turn off blatant spam bouncing

So far I haven’t gone back into this hard bouncing state since the changes but the Postini quarantine contains in the neighborhood of 10 times as many emails as before.

JasperReports XML Datasource with Inline Images

We’re adding XML datasources for reports in RightsPro (currently JasperReports is supported) and needed to embed the image data directly in the XML so thought I’d expand on the solution I found here.

If your creating your XML dynamically using Java you can do:

...
import org.apache.commons.codec.binary.Base64;

// Read the byte array from DB or whatever
byte[] imageByteArray = getImageByteArray();

String base64Image = Base64.encodeBase64String(imageByteArray);

...

then write the Base64 encoded data in your XML as:

...

<image><![CDATA[ ... Base64 mess here ... ]]></image>

...

The rest comes from this JasperForge forum post which explains how to decode the Base64 into a ByteArrayInputStream that’s usable by the JasperReports image report element.

As the post says, in your JasperReport (we currently use iReport as a report editor) define a String field, let’s call it Image, for the image:

then define a variable, let’s call it ImageBytes, which uses that field:

with the Variable Expression:

new ByteArrayInputStream(new Base64().decodeBase64($F{Image}.getBytes("UTF-8")))

(don’t forget to add org.apache.commons.codec.binary.Base64 to the imports of the report properties)

and in the image element on your layout use the ImageBytes variable:

and you should see your image properly rendered in the final report.

However, what I found at this point is that while the images displayed fine in iReport in PDF preview most PDF readers were unable to display the images, resulting in just a black rectangle.

The solution was to change the ImageBytes variable class to java.awt.Image with a constructor of:

ImageIO.read(new ByteArrayInputStream(new Base64().decodeBase64($F{Image}.getBytes("UTF-8"))))

then change the expression class of the image element on the layout to java.awt.Image as well. You’ll also have to add javax.imageio.ImageIO to the imports of the report.

This technique can be useful when you need to grab the image data dynamically (not from actual files on the filesystem), you can’t get each image from a URL (perhaps due to security constraints), and don’t want to use a custom Java datasource.

What to Look for in Managed Hosting

If you find the need to outsource some or all of your server environment during your bid to take over the world, remember that there’s a huge difference between a hosting provider that merely gives you access to hardware, be it physical or virtual, and one who truly manages those servers with you. Here’s a few traits to consider that can prove invaluable not only in day to day operations but when it comes time to upgrade, expand, or add to your deployment:

Level of Competence

[picapp src=”0266/3259b9c3-4908-49f9-9f9e-b8b621911eee.jpg?adImageId=6722375&imageId=269668″ width=”83″ height=”125″ /]

No single provider can excel in every area for which you might need a server. An asterisk VoIP deployment has very different requirements from a LAMP app for example. On the other hand, most apps one might deploy these days require knowledge of quite a few technologies, and most providers will probably tell you they can do whatever you need, so it’s important that you vet the provider’s competency in the areas that will benefit you the most.

Of course the level of competence in any area can vary from complete jackassery, to being able to solve a problem after exhaustive research, to being real innovators in a space, on the forefront of the subject, even disseminating their knowledge through blogging or other means.

A Good Trouble Ticket System

[picapp src=”b/d/5/e/Businessmans_hands_tied_594d.jpg?adImageId=6725332&imageId=5231632″ width=”154″ height=”135″ /]

Phone support is often overrated and inefficient, especially when it concerns servers. A phone call to support is almost always going to spawn a support ticket at some point, requiring follow-ups, attachments, etc. so it just makes sense to initiate it there.

A good trouble ticket system will allow you to email support with your issue, identifying your account with your email address, and will maintain any CCs you included on the original request.

Seamless Handoff Between Techs

[picapp src=”f/f/c/7/closeup_of_the_eabd.jpg?adImageId=6725507&imageId=5159404″ width=”167″ height=”120″ /]

When an issue reaches a support tech on the providers end who might not be the best qualified to handle it, the transition from that tech to another is a crucial step in the problem solving process. The customer shouldn’t be responsible for bringing each new tech assigned to the ticket up to speed.

Great support departments handle that transition so well that you have to double check the ticket to see who you’re replying to.

Specialization Rather Than Escalation

Most of us have had trouble tickets rise through tiers of support where the first is really just gathering information or digging through their knowledgebase for a previous answer, the second has some problem solving skills, and the third or higher are some form of ‘engineering’.

It’s fun to screw with first tier techs and tell them whatever they need to hear to escalate the ticket, but more fun when you get the sense that all of your provider’s support staff are ‘engineers’, and if a ticket does get handed off to another tech it’s most likely due only to a shift change or because the second tech really knows the crap out of the subject and the support team thinks that person would be the best to tackle it.

Response Time

[picapp src=”9/4/9/5/Stopwatch_bf74.jpg?adImageId=6727212&imageId=5288411″ width=”100″ height=”100″ /]

Whether the issue is handled via email, ticket system, instant message, or other medium, response time is key.

Most deployment servers are mission critical (for at least someone’s mission) and downtime of more than a few hours can cause rapid hair loss.

Even if an issue isn’t resolved immediately, some indicator that the provider is looking into the issue gives great piece of mind.

Monitoring

It’s no secret that you and your support staff love dealing with angry users, but it’s often beneficial to have trouble brought to your attention before your users notice. Maintaining resource monitoring and alert systems is a job in itself and a good provider will let you know not just when your server or app has been down for a while but when it’s getting close to consuming its allocated resources, is under some sort of malicious attack, or otherwise exhibiting odd behavior.

When done right the level of monitoring is so good that you feel like a jerk when you forget to tell the provider you’re going to work on your own application and will be taking it down.

They Take Action

Being notified that something isn’t working is one thing, but having a provider that takes obvious first steps to solve the problem, like restarting a service, can save all sorts of time, and may circumvent the need for you getting involved at all.

They Call You

[picapp src=”a/d/b/d/Red_rotary_telephone_0d23.jpg?adImageId=6729532&imageId=5288594″ width=”113″ height=”150″ /]

There are times when phone support does come in handy.

No one likes to be woken up in the middle of the night because an app has crashed and can’t be brought back up, but if the provider’s team has done everything they can without interaction from you and you’ve selfishly closed your eyes for a few hours without a mechanism to poke you for each new email received, a phone call may be the only recourse, and it’s certainly better than waking up to find that a critical service that has been down for hours.

Trust

[picapp src=”0071/1e1b8e53-5aa8-47e9-a2dd-33a62c1c2957.jpg?adImageId=6730842&imageId=74056″ width=”200″ height=”129″ /]

It’s only when you really trust your provider that you can realize the true value of their services.

When you trust them to install and setup a third party application rather than tackling it yourself you save the inevitable hours searching through documentation and forums to resolve some stupid issue unique to your deployment environment.

When you trust their recommendation on which app to use to fulfill some need you can save hours of additional research time.

When you trust that their network, power, and most importantly people infrastructure can handle your app when it becomes the mega success you’ve dreamt it will be, you’re free to dream of the next.

If trust could be quantified and put on a feature list we’d certainly have an easier job of comparing these folks, but as we all know, this one can only be earned over time.

An Unsolicited Shout Out

I can say from experience that Contegix excels in all of these areas. I’m in no way affiliated with them, just a satisfied customer.

Good luck in your search.

Growing jQuery Progress Bars

At RightsPro, we’ve chosen to go with jQuery UI as an interface framework and while the progress bars look great out of the box we wanted to add the effect of the progress growing to its value when the user visits the page to draw attention to it.

jQuery’s documentation for the UI components, including the progress bar, is straight forward.

Once you’ve included the proper javascript files and created the progress div a simple javascript command adds the appropriate classes and creates the progress bar:


$(function() {
$("#progressbar").progressbar({
value: 37
});
});

but to add the growing animation effect we initially set the value to 0 then animate the width of the progress bar value div created by jQuery’s javascript:


$(function() {
$("#progressbar").progressbar({
value: 0
});
$("#progressbar > .ui-progressbar-value").animate({
width: "37%"
}, 500);
});

where 500 is the time the animation should take in milliseconds.

This gives the desired effect of the progress bar growing to its value when the page loads rather than immediately rendering the final result.

Not earth-shattering, but a nice little trick.