Jiri Lundak

Archive for April, 2012|Monthly archive page

ACCU 2012 Oxford – Day 4

In ACCU 2012, Conferences on April 29, 2012 at 8:41 am

There were four lightning keynotes today:

Death by Obesity (Didier Varna)

Very fun talk by Didier showing why certain programming languages will die because of their obesity.

Natural language grow in size (by “vocaculibrary”) because just the numbers of words grow, but grammar remains more or less static.

Programming languages grow in complexity (by syntax) and that is wrong (grammar grows)

Homoiconicity: the LISP UFO (Unidentified Functional Object) is undervalued for its austerity. Languages are easily extensible, because they have very simple semantics and very high homoiconicity (just two language constructs ‘atom’ and ‘list’. All other elements can be created using these two concepts and do not bloat the language itself.

Thinking (Charles Bailey)

Fun talk by Charles, about thinking what you do before you code it. (STOP, LOOK, THINK)

Introduced new pattern: NLP (Named Lambda Pattern)

Geek Feminism (Emily Bache)

Greater diversity in any group makes it more intelligent (Book: Wisdom of the Crowds)

Do not treat woman in software engineering (and anywhere else, for that matter) like aliens.

Find Your Motivation (Jurgen Appelo)

10 Intrinsic motivations within the individual (use it on others and yourself)

Why am I working in this company (3 choices: let it, leave it or change it)

Writing Usable APIs In Practice (Giovanni Asproni)

General API definition: a package, classes, exceptions

Difference between public and private APIs.

Assumption 1: any non-trivial programming creates APIs.

Assumption 2: good code is usable code.

We know different principles: DRY, SOLID, principle of least astonishment

…but not always it is clear to us how to apply them.

Giovanni then cited three people saying something that correlates with good APIs:

  • Fred Brooks on “Conceptual Integrity”.
  • “Make Interfaces Easy to use Correctly and Hard to Use Incorrctly” (Meyers)
  • Golden Rule of API Design (Michael Feathers)

He recommended some techniques for better API design (showing good and bad examples):

  • Put yourselves in the shoes of your users (TDD, if writing the test is painful, then you have a problem)
  • Naming (simplest (one) name for most used concept)
  • Give control to the caller (correct application of SRP implied)
  • Explicit context (deployment (dependencies on othe API, user permissions) and runtime context (preconditions for calling methods, per- and post-processing), be careful with global state)
  • Error reporting (how are errors reported, when, how do we recover from errors, error at one level may not be an error at a different error)
  • Logging as a feature (for support: error and info; for diagnostics: debug and trace), diagnostics should be not in production code: maybe just in your workspace! (give the programmer a choice, maybe he does not need logging)
  • Organisation of modules and classes (start specific and small, YAGNI, later just generalize as needed, high cohesiveness and low coupling)
  • Order of function parameters (keep it consistent)

Two things he mentioned along the way, that made me think:

  • Scala lets you do things in many ways (this complicates the language) and makes its API often unclear.
  • Beware of dependency injection (not always clear, what happens behind the scenes and when, especially with some of the available DI libraries; often seems to be easier to manage things yourself).
Take-away of the day

Languages with a simple and stable core (just a few key concepts and homoiconicity – just learned a new word today – are more powerful, easier to learn and much easier to extend, without bloating them)

ACCU 2012 Oxford – Day 3

In ACCU 2012, Conferences on April 28, 2012 at 3:05 pm
Keynote: Requiem for C (Robert C. Martin)

Great talk – as we are used too – from Uncle Bob. Based on his personal history and the inevitable evolution of hardware to the point, where a single processor is not interesting for the individual developer anymore and thus abstractions (like virtual machines) are mandatory to be able to cope with the increasing complexity of computers. And the family of C languages is to die out, because of its close relationship to the ‘metal’.

Glad I switched to Java and similar languages years ago!

But the example of Uncle Bob himself shows, that you can remain at pace with technological change, but you need constantly challenge yourself. Someone recommended to learn a new programming language every year, but I think once in a while is sufficient, if you want to obtain a bit of a mastery.

As I need to refresh my programming skills, I am thinking about choosing a functional programming language next. Based on that I choose two of the session of this day.

Functional Programming You Already Know (Kevlin Henney)

Kevlin made clear, that Functional Programming is an old concept: it goes back to the 60s (Hoare, Dijkstra, et al.) and was already present in many languages. He presented in an entertaining way, examples of code in all languages at least implementing some of the concepts of functional programming. He did not use any of the new and fancy functional languages like F#, Clojure and Haskell.

Functions are already first class objects in C#, but Java still waiting (wait for Java 8)

An interesting point he made, that in the in some environment the problem of concurrency is solved using a pure data flow approach, which is really simple and much more powerful, than any threading mechanisms, where you have to think about locks and mutexes (think Linux pipes and filters).

SOLID Deconstruction (Kevlin Henney)

Another nice talk of Kevlin, discussed in the context of the Dreyfus Model of Skill Acquisition.

He discussed what the SOLID principles really ment, arguing, that they should be applied more broadly, than they isually are used. So is the SRP valid at all structural levels (program, package, class, method, etc.) of an application.

Interesting thoughts about what Cohesion means can be found at Glenn Vanderberg’s Blog.

Satisfying on Pattern, one might compromise another. So make informed tradeoffs.

Or the LSP for example should not just be based on the syntax of a class, but on its behavior! Interpreting the original definition of Barbara Liskov would mean that a new implementation of some functionality does exactly the same thing as another implementation in another subclass.

…and by the way: going to its original definition the open/closed principle states the same thing as the LSP.

Another intersting thought: Dependency Inversion is nothing more than cohesion in respect to change. We do not want to depend on code that is inherently instable. Thus we make our own code independent of it (for example, by extracting an interface and so hiding the instable code).

In the end Kevlin introduced a new acronym (to the great amusement of Uncle Bob, sitting in the in front of him): FLUID – meaning:

  • Functional (Pipes and Filters, programm divided in self-contained units)
  • Loose (Alan Kay)
  • Unit Testable
  • Introspective (self-observing)
  • ‘Dempotent (aka. Idempotent, gracefully dropping the initial ‘I’)
Complexity Thinking or Systems Thinking++? (Jurgen Appelo)

Jurgen’s talk was fun an gave a quick overview about his book ‘Managemnt 3.0’. Having that book I will not go to explain it here. Just go and read it. It is definitely worth a look.

Take away of the day

Functional Programming simplifies much code, and is definitely a programming paradigm to explore further.

ACCU 2012 Oxford – Day 2

In ACCU 2012, Conferences on April 26, 2012 at 10:33 am

Keynote: The Congruent Programmer (Phil Nash)

Phil started by pointing out the difference between dynamic and static quality. I kind of missed that one, as I was not ready yet to write it down. Have follow up on this though in a separate post and research it first.

Phil then went on to define agility, as “to be able to move easily in any direction”. So we are able to do this we need: poise or good posture (he highlight in this context the seminal work of Frederick Matthias Alexander, an Australian actor, the inventor of the Alexander Technique). He had problems with breathing during his performance, and he discovered, that his problem was caused by problems of posture, especially of his head, neck and spine. With his technique he managed to correct his problem, aligning all three.

As people/programmers we have often similar problems. We want to be agile, but have sometimes a problem of poise or good posture (not only in the literal sense).

As we should keep in line spine, neck and head of the physical body for well-being, the same holds true to our personal, emotional and mental balance.

So we should ask ourselves: “Why am I a programmer?” (or manager or whatever). “Am I guided by a main principle?” (it would be healthful if I was).

What are our beliefs, values and principles? Everyone will have different ones. So…

…write them down, tell people about them, start doing something, see what needs to change. But don’t throw everything away! Make changes one step at a time.

When there is alignment between beliefs, values, principles, that is personal congruence!

Doublethink: simultaneously adopt two opposing believes, this might not be your cup of tea. Act accordingly! Else try the synthesis out of thesis and antithesis, and then live with it.

One strange effect make adaption to new beliefs difficult (to say the least): cognitive dissonance. Because of it we tend to reject (new) beliefs or make a strange mix of beliefs, just to feel better. It is also responsible for the so-called “black and white thinking”.

Some pointers to what explore:

  • On how to resolve opposing forces: Benjamin Michtell (Chris Agris)
  • Spoken theory vs. theory of practice
  • Time out – self-discovery
  • Ask others to assess you
  • Look for opportunities (Example: Mike Lee (was at Apple, grew dissatisfied with life, went to Amsterdam and founded Appsterdam)
Lean Quality Assurance: Much more cost-effective Quality Assurance methods than Testing (Tom Gilb)

Tom stated: Quality assurance is far more than ‘to test’. Quality is far more than ‘absence of bugs’. He did not just make the claim, but proved it with numbers, based on Capers Jones’ longstanding work, grounded in real numbers:

He measured the effectiveness of of finding all possible bugs (100% = all bugs found), for example…

…regression tests: 15 – 30%

…integration tests: 25 – 40%

What about inspections (that actually are used to reduce cost to reach a certain quality before release)? They range at about a maximum of 60% (have to check the real numbers for these)

So what are we to do? We could do the following:

  1. Design Quality in: reuse a high quality product, instead of doing it yourself
  2. Prevention: change root cause of quality problems (the process, refer for this to Tuesdays tutorial)
  3. Inspection: catch bad information from which programmers write bad code (the requirements)
  4. Combine Best Practice Testing (but know, that there still remain bugs, when we seem to have found the last ones, so there is little hope of “zero defects”: between 8 and 10 control stages are necessary to achieve to catch 95% of defects, before going to production

By the way: Bugs are a bad measure of quality. That is why engineers use for example Mean Time Between Failure (MTBF) as a measure of quality.

David F. Rico has some interesting information up on his site at http://davidfrico.com about the ROI different methods of software development approaches have (including Agile processes). Have to check that out.

Real Teams (Henrik Berglund)

Negative example: Skylab 3 Crew went on the 27.12.1973 on strike in space, because they had an exactly timed day, crammed with experiments, with total control and observation from the ground, with any personal freedom.

So what helps teams to be sucessful?

  1. Clear authority to manage own work process (Delegation Poker, use Group Wisdom)
  2. Compelling direction (orients and energizes): product vision, release vision, sprint goal
  3. Challenging work: purposeful with at least a 50% chance to succeed
  4. Clear boundaries: who’s on the team and who is not, no partial assignment of a 20% person to the team, that is 80% in another team, nobody can succeed until everybody does, assign problems to teams not broken down user stories (do not let the POs get between the developer and customer)
  5. Trustworthy feedback on results (as soon as possible)
  6. Team composition (size, differences, interdependent skills, team skills, teams know best whom they need, let them decide who is on their team)
  7. Do not hold back your opinion to avoid conflict or hurting your co-worker’s feelings (Innovation synergy based on open positive conflict based on trust)
  8. Keep team stable for some time
  9. Team building is not rock climbing, but conversation within the team about goals, motivation, etc. (techniques: design a product box, chart: “what does everyone enjoy / bring to the team”, team goals excercise: “what teams were you in”, excercise to find individual coals: “after I have been in this team, I have done [something fullfilling]”)
  10. Team decision making: tool #1: fist-to-five, use for team charter or any other thing that has to be decided by the team (quick poll)
Groundhog Day of a Team Leader (Zsolt Fabok)

What does make a good team leader / mentor (note: this is not necessarily an Agile view)?

His Self-view:

  • Presence (he is visibly there)
  • Trust (can be trusted)
  • Craftsmanship (can set an example)

His view of people:

  • State-of-mind
  • Motivation
  • Mentoring
  • Empowerment
  • Do the dirty job (for example fire people)
  • Show Example (if you have a new idea: live it, own it, promote it, as you are the innovator)

 

ACCU 2012 in Oxford – Day 1

In ACCU 2012, Conferences on April 26, 2012 at 12:30 am

I was a bit undecided on what sessions to follow. With 5 parallel tracks you can end up with a time slot with 4 talks competing for your attention, versus time slots, where non of the talks really sounds interesting. So here my choices for the day:

Keynote: Project Patterns: From Adrenaline Junkies to Template Zombies (Tim Lister)

Tim – as always – had a refreshingly simple yet powerful message: Observe your project and your organization, try to distill recurring behavior patterns, reflect whether they are a good or bad sign and take action accordingly. He presented some patterns from his book and they found wide acceptance with the audience. Recommended!

Git in the Enterprise (Charles Bailey)

The content of this talk was quite technical, outlining for example ways of authentication and how they influence the possible workflow within an enterprise. In the last third of the session Charles gave some advice what to look out for, when implementing Git in any organization, when it comes to branching and merging back into a possibly stable head.

Unfortunately his talk was peppered with “ehm”s, so that the flow was compromised. Also his slides were very dull, missing to visualize what he was talking about.

The Victorian Exploring Expedition of 1860: On Project Management and Leadership from a Real-Life Deathmarch Project (Jim Hague)

Jim told a wonderfully detailed and well-researched story of an expedition trying to venture as far north in the – at that time – unknown Australia of the mid-19th century, doomed to fail from its inception.

The appointed leader of the expedition, although capable of some form of leadership, committed almost every error one can commit in preparation and in execution of the project. Inexperience, bad preparation, absence of risk mitigation and sometimes plain ignorance and arrogance, made the project fail, although equipped with more than abundant resources at the beginning of the journey.

It was easy to see parallels to software development effort, so that all in the room were immediately sold on the reflections presented.

Database Development using TDD (Chris Oldwood)

Chris gave crisp and clear talk about how to apply TDD principles, when working with relational (SQL) databases. He proved, showing a simple example, how one can go about, building data structures, that have a solid set of tests, that express the clients requirements for the data structure.

Some of the points, that stood out of his talk:

  • Encapsulate your database (create a public interface and only expose that interface to your clients, as you would do in your normal code)
  • Test only publicly observable behavior (as exemplified by your public interface)
  • Helpful practices: use schemas for partitioning your data model, embrace composition (stored procedures using other stored procedures), single responsibility principle (as known from OOP) per applied to stored procedures
  • SQL unit testing: SQL Units (procedure, function, view, legacy (constraints, triggers)), tested writing SQL code.
  • Development Sandbox: Isolation, fast feedback, deterministic, tooling is an issue (SS-Unit), as not much commercial tool support does exist.
Takeaway of the day

I had some chat with people during the breaks. And most of them knew what Agile is about, some even trying officially to do Scrum, but most of them are not near implementing it to full effect. Can it be, that we are one of the few companies doing it seriously? And this more than 10 years after the manifesto was formulated. Hmm…

 

Putting the “Engineering” Back into Software Enginering

In ACCU 2012, Agile, Agile Development, Architecture, Design on April 25, 2012 at 12:01 pm

Yesterday I attended a whole day pre-conference tutorial with the late Tom Gilb titled “Advanced IT Design and Architecture” at ACCU 2012 in Oxford, UK. As Tom pointed out, he would have preferred to give the two-day version, because of the sheer mass of material covered (including lots of material to study at home, including an electronic copy of his book “Competitive Engineering“, that does not fit on a memory stick under 1GB).

His basic message was, that “real architecture” needs quantified requirements, derived from concrete stakeholder needs, as its foundation. These must allow to be numerically evaluated to help drive making informed design decisions.

He presented some simple tools that help in the process:

  • A formal specification language named “Planguage”, to be able to arrive, using decomposition, starting from unquantifiable ambitions for any particular needed quality, like “data consistency”, at quantifiable requirements (each with its own numerical scale), for which a current baseline value is stated and “tolerable” or “wished for” goals are defined.
  • Design Impact Estimation: a structured way at trying to come up with a matrix of weighted (sometimes partial and even complementary) design strategies, to help with the choice and prioritization of the alternative(s).
  • Evo: an Agile process that allows to get early feedback on design attributes, before scaling up and committing to a certain design.

During the day Tom multiple times attacked the Agile community as being ignorant to the fact, that design and architecture are fundamentally important to a successful project. He illustrated his point with some example from failed “Agile” projects.

As a long-standing proponent of Agile software development myself, I feel obliged to reflect on the state of affairs and to respond.

The Relationship Between Agile and Architecture

Here are some articles (pointing partially themselves to further articles) that tried to highlight the relationship between Agile methods and architecture:

Sure enough you will find other articles as well, many of which are part of the IEEE electronic library.

In the Agile community we have in practice a very ambiguous relationship with architecture. There seem to be people and organizations, as observed by Tom, that use the notion of “no big design up front (no BDUF)” as an excuse to do no architectural or design activities whatsoever. I challenge, that these individuals or groups of people, would not architect and design their applications properly, even when applying traditional processes. As Tom also stated, most of so-called “architects” today just throw acronyms at you, without ever working like engineers.

So this is not a problem of Agile, but of human nature. The only allegation one can make to the Agile movement is, that it does not properly address this issue with a set of good principles and recommended practices, that are as easy to apply as possible, but at the same time as rigorous as necessary. What happened with automated testing (unit testing, TDD, BDD, ATDD, etc.), that it spread like wildfire, even to non-agile organizations, should also happen with good architecture practices.

The Missing Link

People trying to write good software that has real business value to the customer, will do whatever is necessary, to deliver it. In an Agile environment, they will try to architect and design (in the sense Tom uses the words), as much, as is absolutely necessary, and not more. Recognizing, that a system will need to respond to 300'000 user requests per second is not unnessecary information for an Agile development organization. When I know this in advance, it is wise to choose my design alternatives accordingly, because knowing this essential requirement limits my solution space.

As I perceived it, Tom did not propagate the one and for all cases perfect BDUF, but he installed something more important: a toolset that can be used to define and measure non-functional requirements, that affect our systems architecture. This does not preclude or even prescribe any particular design solution, but gives us a way to validate, how good our chosen design satisfies these requirements, something that goes often forgotten in an software engineering effort, be it Agile or not.

Where to go from here?

To make the adoption of quantitatively measurable non-functional requirements more probable and sustainable, we need to be able to make measurement (whereever possible) automated and iterative, so that changes in requirements are recognized and taken care of and changes in the architecture and design of our systems (especially if they have more of a product character) can be measured against the fulfillment of the stakeholders' expectations.

There is still some serious work to be done here.

 

Competence Development

In Career, Competence, Job, Mastery, Personal Development on April 5, 2012 at 11:49 am

Sometimes I feel hopelessly losing grip on things I mastered in the past. And this frustrates me. The more I am pressed in the manager role, the more I feel uncomfortable. I know I should not. This is social ascent after all. But I am not satisfied. But before taking any action that goes in the wrong direction, I need to asses why I feel this way.

So I tried to put my thoughts in a graph. Let’s call it a competence development chart:

It shows 5 kinds of competence:

  • Social: This is the type of competence we begin acquiring in kindergarten. Most of it is ingrained in us, before we even start working. This is also a quality we train every day. For me this includes talent to be a leader and a good communicator.
  • Technical: As a software developer, this is your primary and most important field of expertise. It is also the most easy to loose. Technology evolves quickly, your code base moves on and your coding skills rust. Without constant exercise and application you immediately fall behind and it’s hard to make up lost ground.
  • Domain: Domain knowledge is an asset, that can make you invaluable in turning business needs into working software. It helps you asses the necessity of a given feature and lets you make informed design decisions. You will not just be coding away on something somebody else specified, but your are truly able to help creating the product the customer really wants. Usually you will immerse in multiple domains during your career. And although you will be able to transport part of your knowledge of one domain to another, you will be mostly starting over.
  • Business: Ability in the business domain does not come naturally to everybody. Not all people have an affinity for doing business on their own. But when you aspire to a holistic view of software development or you believe to have that great product idea or just want to sell you service, or when you feel that business is limiting you as a developer, then this can become a driver for acquiring some business knowledge.
  • Holistic: This is the ability to connect all the dots. You have a natural feel of how things should work, why things go wrong and what could help. This sets in, when you acquire competence technically, socially and at least in one domain. As all competences “seeing the big picture” can be lost over time and peeks, when you are a master in the three previously mentioned areas.

What happens, when you have just competence, that is limited to one field? Let us have a look:

  • Technical: When you have development skills only, you can develop anything based on a specification. You can not know whether what you implement is fit for the customer’s task at hand. You need somebody else to tell you. Some middleman or the customer himself. If you add a middleman, then you can not be sure, the what you implemented will really match the customers’ needs. Rework is preprogrammed. Something you do not like to do.
  • Domain: If you have only domain knowledge, then you might be able to describe to a developer what things you would like to have implemented. You are not necessarily able to explain how it should be implemented. But as soon as you are shown a solution you immediately can tell if it fits your needs. You might build over time an opinion on how the current system you are using works under the hood, but you never can be sure. Especially if developers are allowed to alter the system. So you ever so often will have to adjust your mental picture. And you will need a developer to inform you on how it really works.
  • Business: If you have business knowledge only, you are good at making financial decisions, at formulating contracts, at negotiating with customers, partner companies and vendors. You might have a flair for leading people, but many do not. You are not able to judge the work of developers or domain experts. Even if you had previous technical or domain knowhow, it can quickly fade away. You depend a lot on other people creating value. A problem you definitely have is, that you depend mostly on other peoples’ information, that often was already altered, adjusted and watered-down, so that it does often not represent reality anymore. And you do not even posses the ability or time or both to analyse or validate that information.
  • Social: If you are good only at connecting (with) people, you might be able to use this ability to be the glue to bond teams or groups within an organization, but not much more.
  • Holistic: This kind of competence can not exist in isolation. The more competences we are able to maintain (nearly) simultaneously, the closer we can get to seeing the whole, the better we are able to develop and maintain a global understanding, that does not get lost in detail.

Now I do not want to underrate any person having an ability just in one of these areas. There are only a few people having deep knowhow and understanding in all of the areas at the same time. But most of us do have some knowhow in all of them. And we can try to maintain that knowledge crisp, clear and current. How is it possible to maintain a sufficiently deep knowledge in more than one area?

Below are the ways I currently see. You might have other possibilities you want to share.

  • Concentrate on one area: I should do one job and do it well, at any one point in time. Trying to be a “jack of all trades” usually does not scale and does not benefit anybody, not even myself.
  • Do things in parallel: Filling in multiple roles at the same time is not for the faint of heart. Although you might be able to have a bit of a synergy (when you have a management role like head of development and Product Owner at the same time). But much more you will be torn between roles and you will be a mediocre boss and a mediocre PO at best. And if you go down that path, but are a responsible person at the same time, you will be doing lots of overtime, and your private life will suffer.
  • Rotate jobs in shorter time periods: There should be no fear to loose status or clout or position. If I rotate into and out of jobs, like Product Owner, Software Developer, Head of Development or Business Person, then I acquire only additional leadership potential, because I know more than others at the place I work. I will be able to fill many roles, that need to be filled. I can leverage my knowledge to lead in any position. And best of it all: I will be a person, that can not be easily replaced, and should I be replaced nonetheless, I will not have a hard time finding another job.

You might have noticed, that I did not include a leadership or management area of competence. Here is why: I see leadership and management skills always in context of real work. Situational leadership is key. Anybody in an organisation is able to lead, if he is an expert in his field and is interested in making a substantial contribution to the whole. Managing should also be situational and at the same time very collaborational, no one person should think is is able to manage the whole.

So what about mere management positions? They can exist in an organisation. But should the person holding such a position have mainly management skills and not much more? I believe the contrary: they should have primarily active competence in any of the other fields and management skills should be only seen as an add-on. Management competence falls for me in the holistic and social competence areas. Standing on its own, it tends to develop a life of its own, which often is not healthy for the organisation in its entirety, although the one managing might be convinced that it is.

I just made this up for myself. I am quite sure, that my thoughts are still kind of ruff and incomplete. I might elaborate on my actual choice in another post.