Jiri Lundak

Archive for the ‘Architecture’ Category

Putting the “Engineering” Back into Software Enginering

In ACCU 2012, Agile, Agile Development, Architecture, Design on April 25, 2012 at 12:01 pm

Yesterday I attended a whole day pre-conference tutorial with the late Tom Gilb titled “Advanced IT Design and Architecture” at ACCU 2012 in Oxford, UK. As Tom pointed out, he would have preferred to give the two-day version, because of the sheer mass of material covered (including lots of material to study at home, including an electronic copy of his book “Competitive Engineering“, that does not fit on a memory stick under 1GB).

His basic message was, that “real architecture” needs quantified requirements, derived from concrete stakeholder needs, as its foundation. These must allow to be numerically evaluated to help drive making informed design decisions.

He presented some simple tools that help in the process:

  • A formal specification language named “Planguage”, to be able to arrive, using decomposition, starting from unquantifiable ambitions for any particular needed quality, like “data consistency”, at quantifiable requirements (each with its own numerical scale), for which a current baseline value is stated and “tolerable” or “wished for” goals are defined.
  • Design Impact Estimation: a structured way at trying to come up with a matrix of weighted (sometimes partial and even complementary) design strategies, to help with the choice and prioritization of the alternative(s).
  • Evo: an Agile process that allows to get early feedback on design attributes, before scaling up and committing to a certain design.

During the day Tom multiple times attacked the Agile community as being ignorant to the fact, that design and architecture are fundamentally important to a successful project. He illustrated his point with some example from failed “Agile” projects.

As a long-standing proponent of Agile software development myself, I feel obliged to reflect on the state of affairs and to respond.

The Relationship Between Agile and Architecture

Here are some articles (pointing partially themselves to further articles) that tried to highlight the relationship between Agile methods and architecture:

Sure enough you will find other articles as well, many of which are part of the IEEE electronic library.

In the Agile community we have in practice a very ambiguous relationship with architecture. There seem to be people and organizations, as observed by Tom, that use the notion of “no big design up front (no BDUF)” as an excuse to do no architectural or design activities whatsoever. I challenge, that these individuals or groups of people, would not architect and design their applications properly, even when applying traditional processes. As Tom also stated, most of so-called “architects” today just throw acronyms at you, without ever working like engineers.

So this is not a problem of Agile, but of human nature. The only allegation one can make to the Agile movement is, that it does not properly address this issue with a set of good principles and recommended practices, that are as easy to apply as possible, but at the same time as rigorous as necessary. What happened with automated testing (unit testing, TDD, BDD, ATDD, etc.), that it spread like wildfire, even to non-agile organizations, should also happen with good architecture practices.

The Missing Link

People trying to write good software that has real business value to the customer, will do whatever is necessary, to deliver it. In an Agile environment, they will try to architect and design (in the sense Tom uses the words), as much, as is absolutely necessary, and not more. Recognizing, that a system will need to respond to 300'000 user requests per second is not unnessecary information for an Agile development organization. When I know this in advance, it is wise to choose my design alternatives accordingly, because knowing this essential requirement limits my solution space.

As I perceived it, Tom did not propagate the one and for all cases perfect BDUF, but he installed something more important: a toolset that can be used to define and measure non-functional requirements, that affect our systems architecture. This does not preclude or even prescribe any particular design solution, but gives us a way to validate, how good our chosen design satisfies these requirements, something that goes often forgotten in an software engineering effort, be it Agile or not.

Where to go from here?

To make the adoption of quantitatively measurable non-functional requirements more probable and sustainable, we need to be able to make measurement (whereever possible) automated and iterative, so that changes in requirements are recognized and taken care of and changes in the architecture and design of our systems (especially if they have more of a product character) can be measured against the fulfillment of the stakeholders' expectations.

There is still some serious work to be done here.

 

Interesting Thoughts About Architectural Evolution

In Agile, Architecture on July 9, 2007 at 2:28 am

Tom Ayerst, whom I met when becoming a Certified ScrumMaster in Vienna in 2004, writes here some interesting notes about architecture evolution, like:

“What does this mean? The ideal architecture is as described above simple, and focused, it should also be balanced and not restrict you from taking it in any a particular direction. This last desirable trait is invisible, you don’t know you cannot go in a particular direction until you try to go there.”

and

“Over time your architecture will become less balanced and amenable to change. Getting the balance back will probably require an architectural review of some kind and some kind of more disruptive change than another simple refactoring.”

I think he is quite right. I can observe this with our own application framework we use at work. No simple refactoring will do. After multiple refactoring sessions done often as a pair, we are still trying to bring the architecture back on track. It all depends on how far away you already are from the ideal line.

The 3 Axes of Agile Architecture

In Agile, Architecture, Programming on April 30, 2007 at 11:10 pm

When thinking about my last post today, I was not quite satisfied with it. The values seamed quite unfocused to me in hindsight.

Actually I think we need to condense – especially the values – some more, so that we get at the essence of the agile spirit we need to be able to measure a good architecture against it.

In the end I think it boils down to the following three axes, along which an architecture (maybe embodied by a framework) can be judged (see the figure below).

Agile Software Architecture Principles

I would contend, that an architecture satisfying all three of the axes is THE agile architectural basis for software development that is able to deliver on the agile promise.

Let us have a deeper look at the individual axes:

Change-Encouraging

This axis focuses on embracing change. For an architecture to support this, it needs to be easily modifiable, extendable, refactorable. If it is hard to change, it will lead – over the long run – to clumsy code, horrible work-arounds, lots of glue code and very extensive documentation, that needs to explain all of its quirks and potholes.

If it is easy to change and the developer is free to change his application at will, without too much effort, without breaking existing functionality of his application, then the overall design of the architecture (and the framework eventually embodying it), will remain simple, clean and easy to grasp over quite a long period of time (good tending practice implied).

Quality-Enforcing

Agility has one of its foundations in the delivery of production quality code on from iteration 1 to whatever number of iterations a project takes to complete. As such an application developed in agile manner needs a foundation, that has quality build right into it. This kind of architecture makes it hard for the average developer to make stupid mistakes.

It simple does not allow to circumvent good coding practices. It allows freedom of interpretation only in dimensions, that are in the application domain. It helps the developer to test his code in an easy, dare I say fun fashion, so that he wonders, how he could ever live without the safety net he is constantly weaving.

Value-Producing

Though I mention this axis at the end of this text, it is the focal point of agile software development. When we do not produce the right thing, it is of no use to produce it as fast as possible with the highest possible quality.

That said, it is crucial to produce value for the customer. The architecture should not stand in the developer’s way to produce some useful application. It should encourage him to produce value out of the box, to be able to show his customer a thin, but fully working, slice of the final product he will get.

At the same time it should prove to be a tool in the developers hands, that allows to try out easily alternatives, that can be quickly transformed into a visible artifact to put in the hands of the customer, so that fast feedback can be harvested, which eventually flows back into the development cycle.

Complementary products like acceptance testing support and generic self-documentation should help to produce value faster.

Conclusion

To produce an architecture (embodied in a application framework) that supports these essential values, may not be easy, but at least this is a starting point to think more deeply about principles that support this value-system.

As a nice side-effect I now have a system, that allows me to have a look at existing application development framework and weight them against each other. But I will spare this for another post.

Agile Software Architecture Principles

In Agile, Architecture, Programming on April 30, 2007 at 1:30 am

It is not a secret that I am fond of Agile software developement. I have published multiple articles over the last few month on pitfalls with Agile implementation and I am even writing a book about it.

But in my chest there still beats a heart of an architect and developer. As such I would like to see still better tools, frameworks and development practices. Last friday I was the last standing man at our company, with a colleague of mine and we were fantasising about our personal take on a lean, mean application development framework. We found quite a common ground, when it came to our dislikes with the existing way to develop software in the Java world.

It is always easier to articulate your dislikes, instead of reflecting on the values and principles, that you would like to see flourish.

So what we dislike about existing frameworks boils down to the following short list:

  • Configuration in monstruous XML configuration files
  • The jungle of XML WebServices as the SOA silver bullet for all application integration
  • Way too much code generation in the past
  • Although we are fond of metadata to be leveraged in applications, we dislike too much annotations (obscuring further how things work)
  • Writing still applications for the single VM, not using the power of many distributed ressources
  • Frameworks, that grow to such giant dimensions, that you have to install a 60 MB package, of which you use only 1% in your application
  • Frameworks, that want to be everybody’s darling, by including hunderts of alternatives (we integrate anything) for each module and for this sacrifices ease of use and efficiency in programming
  • Generation of boiler-plate code seen as THE holy grail in increasing productivity

When going through these points in retrospect, I would like to synthesize some values and principles, that I would like to see in a framework, that really helps me concentrate on giving a customer what he needs, without much of the grieveance included, when using today’s breed of tools.

Here are the values I would like to see embodied by an ideal framework:

  • Efficiency
  • Purity
  • Easy Modifiability
  • Expressiveness
  • Openness

These should be support by principles the framework should enforce and follow:

  • Ease of use
  • Transparency
  • Scalability
  • Maintainability
  • Testability

There is some tendency towards easing the pain of development with the advent some of the newer frameworks for web development, maybe started by the hype surrounding Ruby on Rail. Other new frameworks, that fit in this category are SEAM and Grails.

While these frameworks tend to point into the direction I would like to imagine, they are still not there, yet. Too much boiler-plate code, sometimes too much XML. And if all else fails: I could still argue, that they are not Java frameworks! 😉

I just would like to take the Agile thought to the extreme and make it as easy as possible (in Java) to implement and maintain an enterprise-ready application. “I know I’m a dreamer…but I’m not the only one”, to speak with Lennons words.

But who knows, out of idealism, sometimes something new and revolutionary can emerge. Let’s get surprised.


Technorati : , ,
Del.icio.us : , ,
Ice Rocket : , ,

QCon Day 3

In Architecture, Technology on March 15, 2007 at 1:39 pm

This is the first real conference day, as the last two were mere tutorial days. So here small report one the day’s session from my perspective:

I opted for concentrating on architecture in the morning, after the keynote of Amazon’s CTO Werner Vogels, on how Amazon’s single web application (yes, the famous online book shop) evolved into a application development platform that Amazon now is opening and making available to the software development community.

What stood out of his talk? Four things:

  • To scale: No direct access to the database anymore. Instead data access is encapsulated in services (code and data together), with a stable, public interface.
  • To decouple: Services are aggregateable from other services or very “thin” web application, allowing to mesh different services together and leave out, what you do not need, so supporting many different applications and uses.
  • One small team own a service in all its aspects. This has the advantage to make the team responsible for the functioning of the service and at the same time giving it the freedom to do whatever it takes to implement it and make it work.
  • Scale later. It is soooo difficult to make it right, that sometimes the effort to do it up front is not justified. Or leave it to somebody, that has the knowhow and has done it already…like Amazon (remember S3 – Virtual Disk, etc.).

Then Kevlin Henney introduced as host the architecture track, talking about architecture quality attributes, something I need currently also look at.

The session of Martin Fowler talking about “Modifiability”. His approach to hold the session, made him even more sympathetic for me. He downplayed his role as Agile fore-thinker by taking a backseat during the session and let instead five of Thoughworks’ senior software designers (or architects, if you like) illustrate different ideas around evolutionary architecture.

More detailed notes on this will be published on these pages as soon as I have transcribed them.

In the afternoon I attended the “Performance & Scalability” talk of Cameron Purdy of Tangorsol. Besides that he put up just too many slide for a talk of one hour (more than 70) and his talking stile is somewhat monotonous, his talk was very information-rich. Especially I liked his list of architectural “Don’t’s”. Recognized several of them in our own architecture on the current project we are doing. We really have to throw out the legacy code, that is lurking in all corners of the current framework we are using.

In one point Cameron made somewhat contradictory statement in contrast to the keynote of Werner Vogels: Architect for performance and scalability up front, because it will cost you way too much to put it in at a later point in a project. So who is right? Well, when I try to consolidate those two talks overall message I come to this conclusion: Include thoughts on performance and scalability in your initial draft of your architecture, but be ready to revide it and to optimize it at a later point, when you know a.) more about the real requirement for your system, and b.) when you are able to see with your own eyes (or mesure), where the effective bottlenecks hide.

The next talk was again by Werner Vogels, about “Availability & Consistency”. A very interesting talk, that needs further digging, because there a lot of ideas essential to high end web applications. More on that in a later entry.

The last talk was by Peter Sommerlad on “Security Patterns”, based on his new volume of the famous POSA (Pattern-Oriented Software Architecture) books. Unfortunately he was not able to present the theme in an involving manner, that would spark more interest in the auditorium. Partly this can be traced to him not being a native English speaker, so he was sometimes having difficulty finding words and he often lost the flow that is so important to a talk. At the same time he bothered to explain far an long what patterns are and instead of analyzing thoroughly only two pattern (an easy one and a complicated one), he wanted to give an overview over the whole book. An impossible task to accomplish in one hour. Too bad. It seamed more like a sales pitch, than a session, people can actually take something away. I know he can do better, especially in German.


Technorati :
Del.icio.us :
Ice Rocket :

QCon Day 2

In Agile, Architecture, Programming, Technology on March 14, 2007 at 3:33 pm

On the second day I participated in Kevlin Henney‘s “Hands-On Agile Development Workshop”. You may ask yourself, why as a proponent of Agile development and coming from a company, that tries to employ Agile practices across all its projects I attend such a newbie-event.

Well this is mainly for one reason: Part of my role at Löwenfels Partner is that of a Agile advocat or evangelist. The other part is to coach our teams in Agile practices and values, by leading through giving an example.

So my main interest in this session was to see how Kevlin presented the session. Some things I observed about his session:

What I liked:

  • Organized Scrum-like in four Sprints
  • Complex-enough problem to solve
  • Sprint 0 used to introduce Agile concepts in a very fun and lightweight fashion.
  • Sprint planning/commitment included
  • Strong test-first approach thaught effectively
  • Team collaboration stressed through pair-programming, changing pair, focus on clean interfaces for easier integration of team results
  • Practical stories from real world projects mixed in
  • No dogmatic propagation of Agile practices, but pushing common sense conclusions
  • Real struggling with personality types (people you do not know) when pair-programming and cope with that diversity
  • Introduction of kind of a metric for a good unit test: nearly non conditional logic in the test itself. This leads to more readable and understandable tests, that communicate well the intention of application unter test, a lot like a good a specification.
  • Nearly as much test code as application code to be tested.
  • “By omitting tests we will not be able to write the double quantity of code”.

What could be improved:

  • Kevlin talked very much himself. I at least would have prefered more interaction

And the reaction of the attendants? Most of them were convinced to need to apply the trained practices in their organizations, some even against the current paradigma in their companies.

Would I recommend the workshop? Definitely! We should do something similar in our organisation (at least for new employees, if not all). I will try in the coming months to build on that.


Technorati :
Del.icio.us :
Ice Rocket :

QCon Day 1

In Architecture on March 14, 2007 at 2:15 pm

Monday was my first day at QCon 2007, the new over-spanning conference on software engineering. I attended a whole day tutorial given by Larry Constantine named “Usage-Centered Software Design: An Activity-Based Approach”.

This was to be a very interesting session, because I was very curious to see what Larry thought on the integration of interaction modeling into Agile software development processes. This is an interesting problem, because there are voices, that contend, that UI design is hard to implement in an Agile manner, because the UI designer needs to analyze the complete set of requirements to come up with a consistent UI, that supports all possible usage patterns the customer expects.

When following Larry’s explanations the problem interaction experts have with the Agile way to do things became apparent.

Larry advocates a usage-centered approach to UI design instead to a user-centered approach. The difference is, that usage-centered approach focuses on the work to do and to optimize that, instead to put the user primarily at ease and neglecting the efficient solution of the user’s problems at hand.

So his motto is: “Put good tools in the user’s hands, that he can creatively solve his daily work problems.”.

This I would say alines very well with the Agile approach to develop software, because we also focus on what problems the user wants to solve in a as easy way as possible.

One thought that alarmed me, was that Larry said: “I am the usablility expert, so I need to involve the user only a bit, so I know what he wants, but I decide, how he gets it.”. Sure, the UI designer knowledge is very important, but it should not remain – at least in Agile development – with the interaction designer. Every team member should have at least a basic knowledge of the field and should build up the rules and guildlines for the UI as it goes on with development, in an incremental and collaborative manner. Up front UI design should be very short and revised as the project goes on.


Technorati :
Del.icio.us :
Ice Rocket :