Drag Old Web Applications Into the 21st Century
June 24th, 2009 by Benjamin Sweet
Do you have “one of those web applications”? You know, it’s critical to your business but it makes you feel like you’re back to using DOS? Maybe it’s something you directly use, or maybe it’s something you’ve linked from your corporate website for visitors to use, but in either case - it’s just doesn’t work quite as well as you’d like, or looks like it’s from the early 90s.
The Problem
In one case a client of ours was using Eclipse, a powerful inventory management system that provided unfortunately only an extremely basic web application portal for customers to login and order products. The web application portal was not well supported by the developer and any enhancements would take 4-6 weeks.
Because the customer facing portal was so basic we were tasked with developing a pleasing, powerful front-end that continued to tie into the Eclipse system. Creating that system was relatively straightforward, being developed in ASP.net with a SQL Server database.
However, we quickly found that as we were able to provide a customized, clean looking interface that the Eclipse parts of the system were found to be lacking. If only we could take the output from Eclipse and modify it before showing it to the user, enhancing it as needed. More modern systems would have a Web Server API or another way to get the data from the system prior to showing it, but unfortunately Eclipse did not.
The Solution
An old developer standby known as screen scraping. Screen scraping, in a nutshell, involves an application making requests to a program and getting results by interpreting the responding content. In this case, we needed to make requests to Eclipse, find patterns in the results, and then recreate the pages before outputting them to the customer.
By implementing a screen scraping solution we were able to fully take over screens that Eclipse had to handle before, and make both design and content changes as required. This could be as simple as hiding some content, but in other cases could be the integration of data from Eclipse and from our own SQL Server database.
Downsides
There are some downsides to screen scraping. One is that if the underlying content changes you will almost certainly have to tweak the screen scraping application. Another is that performance could be slower, depending on the amount of processing on the screen scrape. In our case we did not notice a performance problem.
Upsides
However in this case the upside is huge. When in the past our client was asked to make changes to add a new client, they had to either say “no” or that it would take 6 or more weeks for the change. By screen scraping changes could happen in hours, not weeks - all while maintaining integrity of the system data and offering a more robust web application.
I’m about halfway through the book
With all of the buzz about Microsoft’s latest plight in the search game, Bing, I decided I should do a little research. Naturally where did I start? Google. As Google has become virtually synonymous with web search, Microsoft is fighting an uphill battle even with the troops that they have, but hopefully this will help those who are trying to make a decision a little bit.