Hacking Blazeds to get streaming working on Tomcat

We happened to found that Tomcat doesn’t do blazeds streaming after version 7.0.27. This impacted one of our applications which still runs on Flex front end. Alternative was to use polling instead of streaming though it compromised on the performance. Out of curiosity I started hacking into this problem. Symptom was flex application just hangs without any error or anything. I checked tomcat release notes (http://tomcat.apache.org/tomcat-7.0-doc/changelog.html) and isolated problem up to Coyote jar file. When Coyote jar replaced on the later version this starts to work again. Out of Coyote release notes we found that one change was done to enforce developers not to have connection: close header when you have chunking in place.

53169: Allow developers to avoid chunked encoding for a response of unknown length by setting the Connection: close header. Based on a patch suggested by Philippe Marschall. (markt)

When we observe Blaze code that’s exactly what was happening. https://github.com/apache/flex-blazeds/blob/c7487aa0504d7561276f106c83ff63c7ff66d67c/modules/core/src/flex/messaging/endpoints/BaseStreamingHTTPEndpoint.java

  // Open and commit response headers and get output stream.
 if (addNoCacheHeaders)
 addNoCacheHeaders(req, res);
 res.setHeader("Connection", "close");
 res.setHeader("Transfer-Encoding", "chunked");
 ServletOutputStream os = res.getOutputStream();

There are couple of possible solutions to avoid this problem. 1. Remove close header from chunck-encoding response. While this could be achieved by multiple ways, one clear way would be to make use of a servlet filter. Kudos to vivimice who suggested this hack!

public void doFilter(ServletRequest request, ServletResponse response,
 FilterChain chain) throws IOException, ServletException {
 chain.doFilter(request, new HttpServletResponseWrapper(
 (HttpServletResponse) response) {
 public void setHeader(String name, String value) {
 if (!("Connection".equalsIgnoreCase(name) && "Close"
 .equalsIgnoreCase(value))) {
 super.setHeader(name, value);

With configurations in web.xml:


2. Recompile BaseStreamingHTTPEndpoint.java to remove connection-close header of the chunk-encoded response. You can download the modified version from https://github.com/charithdesilva/flex-blazeds/blob/master/modules/core/src/flex/messaging/endpoints/BaseStreamingHTTPEndpoint.java You can build the whole flex-messaging-core.jar file and replace it with what’s on your application.


Apache Solr Install Notes

I started working on full text searching solution for our map application. No surprise, Apache Solr is the selected solution and here are some quick notes for you to get started with it.

Part 1 – Basic Setup
2. Unzip to: C:\Program Files\Apache Software Foundation\Solr (ensure that you rename the solr download to Solr)
3. Stop Tomcat
4. Copy C:\Program Files\Apache Software Foundation\Solr\dist\apache-solr-1.4.1.war to: C:\Program Files\Apache Software Foundation\Tomcat 6.0\webapps
5 Rename: C:\Program Files\Apache Software Foundation\Tomcat 6.0\webapps\apache-solr-1.4.1.war to Solr.war
6. Open: C:\Program Files\Apache Software Foundation\Tomcat 6.0\bin\tomcat6w.exe goto Java tab and add Java options: -Dsolr.solr.home=C:\Program Files\Apache Software Foundation\Solr\example\solr
7. Start Tomcat and go to: http://localhost:8080/Solr/admin/
Part 2 – Configuring and running the DataImportHandler 
1. Added the following to file: C:\Program Files\Apache Software Foundation\Solr\example\solr\conf\solrconfig.xml
2. Downloaded sqljdbc.jar and sqljdbc4.jar into C:\Program Files\Apache Software Foundation\Tomcat 6.0\webapps\solr\WEB-INF\lib
3.Create data-config.xml in C:\Program Files\Apache Software Foundation\Solr\example\solr\conf\
   Add to data-config.xml:
  url=”jdbc:sqlserver://localhost\\sqlserver08;databaseName=Business Objects”
  user=”sa” password=”sqlserver08″ />
   query=”select OBJECTID,HOUSENO,StreetName,Suburb from Address”>
4. Added to schema.xml (C:\Program Files\Apache Software Foundation\Solr\example\solr\conf) in:
    You can use the interface at :  http://localhost:8080/Solr/admin/dataimport.jsp?handler=/dataimport
Part 3 – Querying 
Faceting:  http://localhost:8080/Solr/select?indent=on&version=2.2&q=house_number:41&facet=true&facet.field=Suburb
You can facet on multiple fields by appending &facet.field= The key to faceting is giving all your documents a category or type.

To be or not to be GWT?

With the current project that I’m working on, I had to work out some GWT stuff to get specific UI level requirements done. In fact I’m not 100% convinced the way it was done that started pinching me to explore more about it.

This is the first time I touched GWT in real live project development. Being java developer for a long time, I have never come across this much of complexity at the UI level especially when you have been dealing with latest technologies and tools. Basically this system has several layers of wrappers before it really feed with back end calls. Also it contains an in-house written configurable UI Binder that uses xml file to configure the UI. According to my understanding this was done to make reusable and scalable UI level components. But I’m not sure if a software house should invest this much of resources to make front end reusable and scalable as these front end tools can be easily replaced by a new technology that is yet to be arrived in near future. As of now, with GWT 2.0 there is a UI Binder feature which allows you to configure your UI in xml files. In fact I found out that smart-client offers you same kind of configurable UI. So all effort they put to make UI configurable is in vain with this kind of releases. Another disadvantage I see is the expertise required to develop the front end. Front end development supposed to be something different so that it requires different mindset with creativity. Obviously GWT development requires experienced java developers, not the creative designers🙂. So are we doing Swing development for the web? I really feel someone should have considered writing a tool to convert swing applications in to GWT type of front end applications which could have added more value, considering the power it brings to the web applications. There are number of cool features that you can enjoy with GWT based front end, providing an end to the all time headache “browser compatibility”.

In my readings, I came across Flex which has similar kind of UI dynamic binding scripts but something easier to deal with. There are so many pros and cons with both GWT and Flex. There is a good comparison here
– More readings.
– Loading speed comparison

Lets make the web a better place

Speed of the web matters everyone who browse over the internet. Internet has come a long way to reach the level of speed that possess today. Still it demands a better and better speed to make possible many great things that are restricted due to slow internet speed. Google has put a step forward to solve this for the world. It’s a great effort, as people who build the web we all should contribute. Please refer: http://code.google.com/speed/articles/

Caching Service

While I was working on my last assignment, there was a need for implementing good caching solution for the company. This has been a long time headache as we did not utilize proper caching mechanism to cache our frequently used data. Only single component that we were using had many draw backs and it was so much restricted cache reservation details only. The highly coupled nature of that component sometime made trouble shooting a nightmare for developers. As I was assigned to work on new searching component, with my refactoring cycles I had to remove most of the highly coupled back-end logics ultimately existing component became obsolete. Therefore I started working on new component and designed it in such a way where we can use it as a service.
This Caching Service has been developed to facilitate data caching needs required not only by our system, but also any JEE based system. When talking about Caching service, a framework is essential to build and govern the infrastructure of it. Therefore I came up with a framework that accommodates existing caching needs with extensibility in mind.

This is a very powerful framework that can be used to implement new caching mechanisms from the scratch or it can be used as an interface for 3rd party caching engine (MemCache, JbossCache, OSCache … etc). It has a user friendly monitoring interface with functionality rich including the flushing results, flush contains by systems connected, view particular requests, delete particular request … etc.
The beauty of this is it has a central control over all systems connected to caching engine and you can govern them individually. You can have multiple systems connected to multiple caching engines, but you can govern and monitor them in a central location. You are free to add new caching solutions written by your own or you can plug new caching engines that use the same set of operations.
I have an idea of sharing something same as this framework to the open source community let’s see what kind of improved version I can come up with.

Heart of the system, Reservation search reengineered

I’ve concluded the new search component for Base system. This has been done in fully scalable manner which aligns with most of our future technical goals. It is important for everyone to understand the concept so we can make sure the new structure would not be affected with new requirements. In order to place the new component with existing structure, several re-factoring iterations were occupied within the same set of objects.
What did I do there?
This is actually very vital question to be answered. Because to the front interface this could be nothing more than retrieving results, may be in much stable manner. But there have been lots of exercises done to ensure the application structure, transaction management, concurrency and most importantly application extensibility. It is important to have an idea about the drawbacks we faced with previous structure.
  • Highly coupled front end business logic
There were bulks of logics identified inside web application tier. These business logic have to be separated and reside inside server application in order to be aligned with the meaning of a layered architecture.
  • Caching mechanism is tightly coupled with the web application
We had our caching component coupled so much with the web application. This kills the reusability of that component and there were no centralized controlling or monitoring capability.
  • Server was not serving properly
We have used server application just to run particular application logic only. Sever had no control or idea about the application flow, it was only managed by the front tier. For example calling thread x and thread Y runs as two different processes. Finally it’s managed by the front tier application which we should have used to process UI tier functions.
I heavily used some refactoring methods to bring this structure to level where I can start re-designing.