Theory of (web framework) evolution

1G
It all started with cgi-bin applications that really did not separate the templating from the rest of the logic.

1.5G
It is a little bit difficult to say if the simple scripting-based approach was any better than the first cgi-bin applications. I say yes, because there you usually had a little bit more structure. The application was divided between multiple scripts, each one usually handling a specific URL. The scripting framework took care of preprocessing the request and splitting, for example, request variables to a suitable data structure.
Examples: Classic ASP, PHP, Perl, Livescript

2G
Developers noticed that mixing logic and markup easily leads to hard-to-understand and debug code. The obvious solution was to apply the Model View Controller that had been successfully used with traditional GUIs. Usually, this led to a programming model where we had one controller per page.

This step did not really require new frameworks. Even the simple scripting solutions allowed one to separate the logic from the markup, although this required some discipline, since both were usually contained in the same script file.

There was (and still is) a great deal of controversy regarding the “how much logic can you have on your view” question. Some people say none, many others are more relaxed.

One big problem with the 2G solution was that they usually provided limited support for encapsulating parts of the user interface (for example, a widget that needs to be on every page of the application). This problem was attacked with different kinds of tiling systems, template level includes (and to some extent with JSP tags). Usually, the problem was that these solutions were just on the view-level - they did not extend to the controller or model part.
Examples: Struts, ASP.net MVC, Django, and many others

3G
The next step was to adopt more principles from the traditional GUIs to the web, this time in the form of components. 3rd gen frameworks allowed users to create components, each consisting of separate markup and logic, and then combining these components into larger components and finally into pages.

These new frameworks made the developer’s job easier for certain tasks, but they were not a suitable solution to all issues. The fundamental problem of web development is that the web is pretty much stateless. This is not just a drawback - it can also make things like building scalable websites easy. But for a fancy component-oriented web framework, the statelessness is a problem - they must manage the state somehow. Some decided to put the state on the client (ASP.net webforms), others kept it on the server. Neither solution made everybody happy, people were afraid of bandwidth use, server-side scalability, or security.

Another issue that came up with the latest breed of frameworks was the lack of control. In order to work, the frameworks tended to mess up more or less with the HTML markup - leading the coder to feel out-of-control. This also caused troubles with 3rd party Javascript components. Including proved to be difficult when the frameworks, for example, took control of the element ids.
Examples: ASP.Net (webforms), Tapestry, Wicket

Bigger is not better
As we have seen with other programming tools, the new generations are not always better than previous ones. Just look at the 4GL programming tools. There are certain cases where they shine, but they did not really take on for the generic application development. The same applies to web development. This is the reason why, for example, Microsoft is seemingly stepping back by introducing ASP.NET MVC. There are simply use cases that are easier to cover with the 2G approach than with the 3G, component-oriented frameworks.

What next?
I believe this is the end for web frameworks of traditional style. Frameworks like Wicket, Tapestry, and ASP.NET (WebForms) will continue to evolve and get better and better, but the fundamental problems won’t go away.

BUT meanwhile, the browser has changed from what it was in the days of NCSA Mosaic. It is no longer just the dummy HTML rendering tool between the server and user that it used to be. Instead, it is a full-blown application execution environment. This is not exactly a new thing. It all started with Netscape adding the support for a simple scripting language, later renamed to Javascript. Then came plugins, Microsoft ActiveX, Flash, Java Applets, and many more. Many of them promised to bring desktop applications to the web. But users and developers did not cheer. The applications written with these did not really feel web, they had all kinds of security and performance issues and compatibility between browsers was usually poor. Simple things like printing became difficult.

One of those technologies has done better and I argue that is the basis of the future for Internet application development. We are already seeing examples of this future when we use products like GMail or Google Docs. The applications abandon the usual model of “click-go-to-server-serve-a-page-wait-click”.

The new way of building applications is completely different from the previous one. Revolution instead of a small step in evolution. But at the same time, it is nothing new. What we are seeing now is essentially the same thing we were doing on the desktops before this great new web way of doing things took over. Back then people were building so-called fat client applications, with the application running on the desktop taking care of mainly user interface-related tasks and the server application taking care of business logic and database access (some people also experimented with skipping the server part and going directly to the database from the client, but this did not prove to be a good idea, so I suggest we skip this part of evolution this time).

The three layers in this “new” model are the same as before (I will take a closer look at these later):

• Client
  Modern web browser, capable of executing Javascript and preferably supporting the latest breed of web technologies (e.g. HTML5). Applications are written either with Javascript or with some other language that is compiled to Javascript. I see no future in other technologies (Flash, Java, etc.) on the browser - except for special use cases.

• Server
  Most, if not all, application servers and scripting solutions used with the 2G or 3G web application development will be usable. Instead of serving HTML pages, they are going to serve XML or JSON via HTTP and pushing information via Web Sockets. The servers are likely to be as stateless as possible, to make things like scaling and clustering easy.

• Database
  Since the database is hidden behind the server layer, there is no need to make new inventions here.