ArticleS
.
UncleBob
.
IncrementalArchitecture
Edit Page:
!title Incremental Architecture One of the questions commonly asked about Agile methods like XP is: * ''What about architecture? Don't you have to have your architecture thought out up front?'' To which I respond: * ''We allow the architecture to evolve.'' And then people ask: * ''Yes, but isn't that just hacking? After all, when architectures evolve, don't they turn into a mess? Isn't that how we used to build systems in the 60's? Didn't we stop because of all the messes we made?'' To which I respond: * ''Evolving an architecture is not the same as just letting the system grow in any shape that's convenient for the moment. Growing an architecture takes thought and discipline. We keep the architecture optimized for the current conditions. When those conditions change we refactor the architecture to keep it optimized.'' Sometimes people draw back in fear, cross themselves, and say: * ''Yes, but then aren't you constantly making sweeping changes to the system? Shouldn't you design the architecture up front to be flexible?'' To which I respond: * ''What kind of architecture do you want? One that was designed up front a year ago and hasn't changed since, or one that has proven it's flexibility by being continuously evolved?'' The most common reaction to this is '''skepticism.''' !3 A big architectural change in FitNesse For the last several months I've been a guinea pig for a significant experiment in incremental architecture. FitNesse, as you are probably aware, is a wiki, web server, servlet engine, and acceptance testing engine (using FiT). One accesses FitNesse pages by using a url of the form: http://www.mySystem.com/MyPage, where ''MyPage'' is the name of the wiki page you want to see. FitNesse is a hierarchical wiki, which means that pages can have sub page. Dots are used to separate sub page names from their parent pages. So you can view http://www.mySystem.com/MyPage.SubPage.LeafPage if you like. Inside FitNesse you can read load a page off the disk by using a class named ''!-PageCrawler-!'' as follows: {{{ PageCrawler c = new PageCrawler(); WikiPage p = c.getPage(root, "MyPage.SubPage.LeafPage"); }}} The ''!-PageCrawler-!'' interface is rich. There are many methods that allow you to access, read, find, or otherwise deal with pages. Most of them take a ''String'' as the name of the page. Several months ago we decided to change the way subpages are named. Instead of ''!-MyPage.SubPage.LeafPage-!'' we decided to use unix path names like ''!-MyPage/SubPage/LeafPage-!''. This change solves lots of problems that we don't need to go into here. However, it left us with one very large problem. FitNesse code is littered with ''Strings'' and "." to construct page names. The PageCrawler interface deals ''directly'' with Strings and dots. We decided that ''!-PageCrawler-!'' should not know anything about Strings or dots. Instead we created a new object named ''!-WikiPagePath-!''. We intend to replace every String argument in ''!-PageCrawler-!'' with ''!-WikiPagePath-!''. You create these objects as follows: {{{ WikiPagePath p = new WikiPagePath(); p.addName("MyPage"); p.addName("SubPage"); p.addName("LeafPage"); }}} We also created a helper class named ''!-PathParser-!'' that you use like this: {{{ WikiPagePath p = new PathParser("MyPage.SubPage.LeafPage"); }}} Clearly, if we can change the ''!-PageCrawler-!'' to use ''!-WikiPagePath-!'' instead of Strings and dots then we can change the way names are formatted any time we like. Indeed, we can use more than one syntax if we so desire. All we have to do is translate that syntax into the appropriate ''!-WikiPagePath-!'' objects. Unfortunately this is a ''big'' change. As I said before the FitNesse code is littered with calls to the ''!-PageCrawler-!'' interface. There are thousands of calls. This is a big architecture change. !3 Incremental Architecture For the last several months I have been slowly making this change. I do it one test case at a time. Indeed, there have been several releases of FitNesse made while this change has been in progress. The reason it is taking so long is twofold. First, it's a big change and would completely consume an iteration. Secondly, I haven't had a lot of cycles over the last few months to put on this. So every time I have a spare hour or two I eliminate a few more of the old type of call, and replace them with the new type of call. How can I make a sweeping architecture change like this and keep the system running and releasable? ''Incrementally.'' I simply write the new version of the ''!-PageCrawler-!'' method; and then rewrite the old method to call the new method, doing the appropriate translation. Then, one by one I change the calls to the old method into calls to the new method. When all the calls have been change, I can delete the old method. At every stage I keep the whole system executing. I keep all the unit tests and all the acceptance tests running all the time. For example, if there are 15 calls to the ''!-PageCrawler-!'' in the ''!-WikiWordWidget-!'' class, then I replace them one by one, running the unit test: ''!-WikiWordWidgetTest-!'' after each change. Then I run all the unit tests for FitNesse, and all the acceptance tests for FitNesse. If they all pass (they ''usually'' do.) I check in my changes. Interestingly enough Micah has been making other changes to FitNesse while I've been working on this change. He's even made changes to the structure of ''!-PageCrawler-!'' and many other areas that I've been touching. We've had one or two minor collisions, but nothing significant. The architechture is incrementally changing. !3 So What? My point is that the architecture of a system can be slowly and incrementally evolved, and that TDD (including both unit tests ''and'' acceptance tests) is the enabler. Without the ability to run those tests and ensure that I have not broken something, I'd be very reluctant to continuously meddle in the guts of the architecture month after month. But since I have the tests, it's a complete no-brainer. I don't mind leaving the architecture half-way between the old and the new, because I know nothing is broken and I can easily pick up the thread of my thoughts by looking at the tests I've written and the current state of the interface. For years now skeptics of Agile methods have predicted that rules like ''YAGNI'' and practices like ''The Simplest Thing'' would lead to architechtures that are stuck in a local minimum. These skeptics predicted that once locked into the local minimum, agile teams would have no way to make bigger architectural changes. For years the agile community has rebutted this by saying that big architecture changes can be done incrementally over many iterations and releases. This change I've been working on in FitNesse is an existence proof. Long term incremental architecture evolution works, and works well. !commentForm ---- 7 Feb 2005, '''Steven Martin''', XP isn't just about hacking. A good team will regularly come together for design sessions, proof-of-concepts then allowing each XP pair to go off on their own. Good performance benchmarks typically take at least 3 rounds to achieve anyway. ---- Far to many developers still regard a design as something static. Having practices like TestDrivenDevelopment and ReFactoring in place changes all that. They make a design dynamic, continously evolving for the better. In the end a 'Big Architectural Change' doesn't turn out to be a Big Architectural Change. It's just a refactoring that takes a bit longer. SvenGorts ---- For years I took pride in my ability to design flexible architectures up-front. Looking back, I realize that I was just over-engineering and typically only used 10% of the flexibility that I built. Now I do it incrementally and I'm not going back! Mike Stockdale P.S. Re: 'XP isn't just about hacking'. XP isn't at all about hacking. XP is extreme discipline. ---- 2005 2 22, Kelley Harris - Incremental Architecture is a great topic. Great topic for a series. Great topic for a book. Examples like this help. More examples would help more. Thanks for writing on this topic. ---- !* Fri, 25 Mar 2005 10:16:15, Dan O'Connor, Architectural Change ?? *! !* Fri, 25 Mar 2005 10:16:15, Dan O'Connor, Architectural Change ?? I appreciate this anecdote. But I question whether this is a “big architectural change”. It is a big change across many files. Is it architectural? It sounds like you wanted to change the delimiters. You found that this had a big ripple affect due to the interface and the large “fan-in” from clients, and so decided to abstract the path/delimiter details from the interface. Maybe “architectural” is relative. But the topic of “incremental architecture” is very interesting. I am interested in additional cases where large systems are incrementally migrated from legacy framework/partitioning/runtime/collaboration paradigms to complete new paradigm in order to leverage modern tools and practices. *! !* Mon, 4 Jul 2005 07:46:49, Richard Henderson, The winding road to the top of the hill. Nice little example. The key point here, IMO, is that you created a gap (adapter methods are nice for that), created the legacy adaptation code, then started fiddling, keeping the interface constant by making counterbalancing changes to the adapter and the adapted. I wonder if this is always possible? In my experience it is, but I'm not sure it is always easy. I'm thinking that 'big' architectural changes such as 'single threaded' to 'multithreaded', or going from 'demand driven' to 'event driven', or partitioning applications across multiple servers etc etc may be somewhat harder. Not impossible, but a lot harder. A gram of planning up front tends to outweigh a kilo of increment after the fact when painful architectural break-points are reached. *!
Hints:
Use alt+s (Windows) or control+s (Mac OS X) to save your changes. Or, tab from the text area to the "Save" button!
Grab the lower-right corner of the text area to increase its size (works with some browsers).