System Performance – Thinking Sideways

I find system performance problems utterly engrossing. Although CPUs have accelerated (in line with Moore’s Law), and multiplied, and although memory has expanded, network communication speeds have increased and access times for storage media have shortened, software has become more sophisticated and we still find ourselves drumming our fingers whilst waiting for our systems to respond.

swirlyscreen

When you’re dealing with a performance problem you must think imaginatively and picture the way computing resources are being deployed. If you’re working on a client-server system, where large numbers of PCs might be accessing the same server software (often a database) it is more complicated still.

Sometimes a server processor is too slow (statistics might show that it is never idle), and processes must wait for its availability. In this case you must try to reduce the work it’s doing, by improving the efficiency of code, or by shifting processing from the server to the client.

Sometimes network communications are too slow, and you must reduce the volume of data being transferred between client and server, or reduce the frequency of transfer (web page size can matter in these circumstances, and program logic).

Sometimes a database is poorly designed and an application must ask too many times for data from too many records. This reveals itself as too much data storage activity, or too much database software processing. The answer, then, is to redesign the database, or configure it more effectively so that frequently accessed data are better cached.

Sometimes there is contention for resources. A large database system will often contain ‘control records’ that ensure consistent use of a system (such as a record holding the ‘next invoice number’, a record that many processes may need to update). If access to such records isn’t managed cleverly, one (or many) processes may have to wait for another process to finish its work. Sometimes the result is ‘deadlock’ when one process has ‘locked’ a record that another process wants, and that other process has ‘locked’ a record that the first one wants.

Instinct, and an ability to imagine these resources vying with each other, is a useful talent when it comes to solving problems. Doctors will attest that instinct plays an important role in the diagnosis of human sickness.

But sometimes you solve one problem and create another. Thirty years ago, when I was a junior programmer, I was working on the development and implementation of a financial management system that used a complex database to store its transactions. The client was exasperated by the poor performance of the system. Analysis showed that the CPU was idle most of the time, whilst the disk storage devices were overheating (metaphorically). I looked at the database design and I could see that it resulted in fragmented, almost randomly distributed data, when smooth physical distribution of sequential data was what was needed.

So I redesigned the database and built a caching mechanism that used an algorithm (using chains of pointers) to retain the most frequently accessed records in memory, where they could also be accessed without ‘locking’. Data access activity fell tenfold, but so busy was my algorithm (I mistakenly used bits rather than bytes as indicators in an unnecessarily sophisticated attempt to save on program size) that the CPU started steaming (metaphorically) instead. The CPU simply couldn’t get through its work. I’d improved overall performance slightly, but insufficiently, and the client abandoned the system. I’d moved the bottleneck from one location to another. There’s always a bottleneck.

But perhaps the most fascinating and frustrating performance problem I was involved in (on behalf of LLP Group) was at Otopeni Airport in Bucharest. Our client, Air BP, was using SunSystems for its financial accounting and reporting (for both local and corporate purposes). They called us, stumped, because, from time to time, and unpredictably, the system slowed markedly and became too frustrating to use. It was a typical client-server system. There were about eight PCs in regular use, and a server sitting in the corner of the office that contained the SQL database that SunSystems used.

We spent hours at Air BP’s airport office. We played around with parameters on their PCs and on the server, and when we were there (as so often happens) the system worked perfectly. It was the exact opposite of the ‘demonstration effect’ which causes software to go wrong just when you want to show it to someone. In this case, it never went wrong when they wanted to show us the problem.

We were stumped too. From time to time we would put our (metaphorical) spanners down and sit and think about it, and have a cup of tea. And usually whilst we were thinking and sipping the system would start to slow down again. Was there a malicious child sitting inside it? We’d go and look at the server parameters, and, hey presto, it started working well again. We looked at the server statistics. The CPU wasn’t busy, the disk storage device wasn’t busy. There was plenty of memory.

And then one of my more logical or imaginative employees had a brainwave.

‘When we’re working on the server, it works well. But when we’re not, after five minutes or so it gets slower. What happens on the server when we’re not working on it?’

‘No idea.’

‘The screensaver switches on.’

And that was it. There was a lovely swirly screensaver that came on after five minutes of inactivity on the server console. So complex and lovely were its graphical calculations that it used 95% of the CPU’s capacity. We simply switched it off.

Sometimes it’s the last thing you would think of.

Beauty on a Small Scale

There’s a lovely new exhibition at the Hunt Kastner Gallery here in Prague of drawings, designs and photographs by Dalibor Chatrny (1925-2011), an influential artist and teacher who studied in Prague shortly after the Second World War, and who spent most of his career based in Brno where he taught, until 1986, at the School of Applied Arts.

The Hunt Kastner exhibition displays collections of designs that Chatrny published privately in small pamphlet editions, as well as photographs, some his own, some by friends and students, that he delightfully ‘enhanced’ with drawn lines and strings. There is great beauty in these small scale works.

In recognition of the profound influence of Dalbor Chatrny on artists in the Czech Republic (and no doubt in Slovakia before the separation), guests at Friday’s exhibition opening were invited to bring their own small-scale works inspired by the artist’s work, together with some explanatory text. Sadly, few complied, and the few works that were contributed by aspiring or established artists (or perhaps complete unknowns) served only to illustrate how difficult it is to create works of simplicity and beauty that don’t need justification through pretentious conceptualist explanation.

Fine works by Dalibor Chatrny

Rubbish (in my opinion) by guests at Friday’s Exhibition Opening

No Dalek

no dalek

Artist David Davros writes: For a child, all is the ‘other’. It is only through a process of secondary familial and public socialisation that a child can construct his or her necessary public and moral structures containing other: LOVE and other: HATE and at the same time (re)configure his or her symbolic understanding of other: TERRESTRIAL and other: ALIEN. The child will all too often and all too easily reach for a palette of anatomically incomplete, sometimes conventional, sometimes degenerative, quasi-distorting representative forms to defuse the power of the hidden multiverse.

Breaking Wave (oil and modelling clay)

breaking wave

I am fascinated, writes Qi Qi-Di, by the relationship between object and representative medium. The wave breaks, just as its representation breaks, cracking and flaking dynamically as the clay dries. And as our cortical synapses progressively fail, and our own perceptions and memories break and fade, object, medium, perception: are conjoined in oblivion. In the contemplation of the unending cycle of our striving and failing to convey, lies our only hope for peace, both in ourselves and in the world.

Out of Box

outofbox

Out of Box, writes Ahmed Oud from ArtOffensiv Kolektiv, challenges the underlying strictures of representation, whether figurative or symbolic. Medium may itself remain untransformed by the assertion of the artistic object. Literally, nothing becomes the object when representation lies outside it. It is a message not through art to the world, but from art to the world.

These eccentric contributions have been incinerated (a condition of their contribution, apparently) so you need have no fear that your pleasure in Dalibor Chatrny’s work will be spoiled by the work of his overly cerebral and practically untalented followers.

Announcing systems@work Version 6

Version 5 of systems@work’s time@work, expense@work and forms@work was released at the end of 2015. It contained a number of enhancements, most of them enthusiastically received and put to use.

Version 6 will be released in early July. It seems soon for a new major release, but the radical revision of the browser-based Professional Services Workbench warrants it. Gone are the variously muddy-coloured lists of functional options, to be replaced by dynamic tabs, panels, links, and shortcuts.

Our aim has been fourfold:

To comply with current standards and graphical fashions
To present the user more clearly with To Do lists for data submission and approval
To enable navigation with fewer clicks
To make navigation pages responses to the user’s device (responsive design)

The result is a Home Page (for time@work) that is more pleasant to look at and easier to use.

Version 6

And list pages that look more modern:

Version 6 will also contain the following enhancements:

A new reporting option that allows you to report on and export static data (such as Employees, Clients, Projects and Tasks). This will be of particular benefit for integration.
Credit Notes in the browser-based PSW
Enhanced emailing options for Invoicing, including an option to select an Employee as a destination and to allow emailing to multiple email addresses
Workflow Control for Timesheets to allow diversion of a timesheet from one authoriser to another

Money Talks. The Dog Barks. Motivation and Vocation

A few days ago the British Government published a Review on Antimicrobial Resistance . It described the growing resistance of bacteria to over-prescribed antibiotics and suggested that by 2050 a death from hitherto treatable bacterial infections might occur every three seconds, unless new drugs are discovered. It also urged huge investment in research, including the offering of large prizes (a billion US dollars) to pharmaceutical companies for every new antibiotic they discover. But these vast prizes are insignificant compared to the cost of medicine being thrown back into the dark ages by recalcitrant bacteria.

It is pointless, as some do, to rail against the rapacious selfishness of pharmaceutical companies and expect them to undertake difficult and commercially risky research without incentives.

The report also urged doctors to prescribe fewer antibiotics, and to avoid prescribing antibiotics for infections that are viral. (Here in the Czech Republic there’s a tendency (admittedly now declining) to prescribe antibiotics as a precaution, to prevent a viral cold from turning into a bacterial cough.)

For many years (perhaps even decades) GPs in the UK have been asked to prescribe fewer antibiotics but are understandably upset when patient satisfaction declines. Nevertheless, data from the NHS were published yesterday that showed that doctors had prescribed 7%fewer antibiotics in the last year than during the preceding year.

How was this achieved? Through financial incentives. Some doctors’ practices were awarded 20,000 GBP bonuses, in effect for doing less.

greedydoctor

Money talks. Or as the Hungarians say – Money Talks, the Dog Barks.

We don’t like to think that financial incentives play an important role in motivating practitioners of vocational professions. The decisions of doctors, nurses, teachers and priests are not, we like to suppose, influenced by personal financial gain. Decisions must always be made on the merits of each case.

But we know that the priest will do us a better wedding or baptation, the nurse’s hands will be softer and the doctor will see us sooner if we oil the wheels a little.

Here in Eastern Europe from 1948 until 1989 society was inspired by the utopian ideal that each of us would eagerly give ‘according to his ability to each according to his need.’ It didn’t work.

We have to accept that money motivates. If you’re managing a company, you must take account of dozens of factors when deciding how to motivate your staff, but if the money is insufficient, none of the other tricks will work – whether you’re juggling with job title, benefits (such as a company car), training, or responsibility. Even if staff say that the ‘money doesn’t really matter,’ they’ll still agree that money is a kind of recognition, whatever uses it’s put to.

You’ve got to accept that money is an incentive, and when you’re devising your incentives you’ve got to be careful. The NHS in the UK, I believe, will reward doctors for diagnosing and prescribing pills for blood pressure, and reward them for not prescribing antibiotics. The rules of the game have got to be sophisticated enough to achieve the right result but not so complex as to confuse, or be beyond efficient measurement. And there are always the unintended, unanticipated effects of motivation that solve one problem and create another.

But sometimes incentives go too far. My mother is currently being hounded by her young GP (probably recently married with a kid to support), who claims she’s diabetic, and, judging by the frequency with which she tells me this, it has disturbed her peace of mind. She’s used a home-diagnostic kit to prove him wrong. Very likely he’d collect a bonus for adding her to the list of Type-2 diabetics, but he’s not going to get away with it.

Money talks. The dog barks. It will always be so. And if it reduces the risk that antibiotics will fail, who cares?

Philosophers for Brexit

I read on the news this morning that Britain’s military establishment (or, rather, Britain’s former military establishment) has come out in favour of Brexit. Dozens of former generals have signed a letter arguing that what matters when it comes to defence of the realm is NATO not the EU.

Historians have come out for In. Actors, artists and other luvvies have come out for In. Economists have come out for both, of course, but what should we expect? It is the fashion for groups of all kinds to hold hands and write to The Times in favour of either In or Out. Where do campanologists stand? Ornithologists? Kleptomaniacs? Nymphomaniacs? Meteorologists? Numismatists? Philatelists? Dog lovers?

philosophers

But another very important group has also nailed its colours to the mast today. Less well reported, but surely of greater import, is a letter in today’s edition of Mind, the journal of the British philosophical establishment, signed by members of Britain’s philosophical community (note that there is no such thing as a former philosopher, unless you mean a dead one). They have come out, albeit quietly, for Out. Entitled ‘But it doesn’t mean anything’ the letter decries the philosophical assumptions on which the EU is built.

Brian Goodlittle, Reader in Philosophical Energetics at Bradford University says, ‘I was approached by the editor of Mind and was asked to sign this letter. I did so enthusiastically. I am fundamentally opposed to the continental drift of modern European philosophy. It favours meaningless nonsense so it’s not actually philosophy at all. I favour the bracing style of British Empiricism. It admits no blather, no metaphysical indecency. During the Second World War British Empiricism was one of the fiercest weapons in our intellectual arsenal. It had few uses on the front line, admittedly, but it helped us to break the Enigma code and, with the help of the Yanks, to build the Bomb, whilst the Nazis were literally dreaming up nonsense. It would be a disgrace if we gave in now to continental so-called philosophies such as phenomenalism, existentialism, structuralism and other forms of poppycock. French philosophy, in particular, is a load of merde, in my opinion. It reeks of garlic and doesn’t make a single iota of sense. Let’s face it, Mate, what does ‘European Union’ mean anyway?’

Another eminent philosopher, Fiona Fruitington, Professor of Radical Empiricism at Northampton University, has calculated that works of continental philosophy weigh on average four times as much as works by British philosophers. ‘Being and Nothingness,’ she says, ‘I would rather read a DIY manual on shelving. EU law is just the same. Voluminous, meaningless and impractical.’

British philosophy has for centuries been tethered to good old common sense. You can only understand a statement if it can be verified, Alfred Ayer told us (though he could never quite explain how this claim could itself be verified). Austrian born British philosopher Karl Popper turned the same idea on its head and said that something only makes sense if it can be falsified (science proceeds that way, he pointed out, rather than by verification, but he never clarified exactly how his own claim could be falsified).

The greatest of them all, my hero Ludwig Wittgenstein, said we must look at how we come to understand language and the meaning of the terms it contains. We must examine language ‘games’ in real human communities (though I don’t think he had the EU in mind). Continental nonsense, and most of what the EU has to say, he would describe as ‘language gone on holiday’, ordinary words extrapolated way beyond their safe and practical usage. The role of the philosopher, he believed, is to show the fly the way out of the bottle, the fly being the ordinary man or woman befuddled by EU terminology.

‘Ever greater union.’ What does ever actually mean, Wittgenstein might ask. How have we come to agree, as a community of minds, on its deployment? And how could we begin to understand the many meanings of ‘sovereignty’ and ‘subsidiarity’?

I have great sympathy for philosophers, but in the end I’m not with them on this. When it comes down to it the vast majority of them don’t know how to boil an egg.

Euphoria

I’ve never felt more glad to be alive than when I survived a parachute jump in 1982. ‘Survived’, perhaps, exaggerates the danger. There was none, we were told, but I was so afraid that I barely slept the night before. To this day, I’m not sure why I, and about four other close friends, did it. What were we trying to prove to each other or ourselves? It might have been because the sister of one of my friends dared us to, or because it was the kind of thing no one would have expected us to do. We were hardly paragons of macho masculinity.

As it happens, that sister of one of my friends was very nearly killed when her fixed line caught beneath her chin as she jumped. So, perhaps, there were dangers after all. The instructor and dispatcher in the plane were immediately fired.

parachute 1

We jumped from only 1,000 metres on fixed lines, so our parachutes opened as soon as we left the aircraft (you can’t be trusted to open your own if it’s your very first time), but it was probably doing it alone that scared me. There were two moments of sheer terror – the moment of stepping, unnaturally, into nothing (this was so terrifying that my mind never registered it at all and it remains a blank), and the moment of landing. The two minutes, in between, of floating gently in summer sunshine towards the green and pleasant Herefordshire countryside, were blissful, though my apprehension grew as the land rose rapidly towards me. The greatest danger for the amateur is landing. A colleague, much older than I was then, who jumped for charity a few weeks later, broke both his legs. I landed without incident.

I have never since felt such euphoria, have never since felt so strongly that I would rather be alive than dead, not even when the dentist gives my teeth a clean bill of health. I understand how people become junkies for danger, and I can only imagine that surviving real danger, when bullets are flying, and bombs falling, is an even bigger kick. War reporters, soldiers, sky divers – I can understand, why they want to do what they do. Even so, I have never wanted to jump into empty space again. Once was enough. Euphoria is probably best remembered, not re-experienced.

My partner and a friend of ours jumped from 4,000 metres on Saturday, at an airfield near Kolin in the Czech Republic, which is what has brought my own experience back to me. They didn’t jump alone. They were attached, separately, to professional parachutists (and, it seems, skilful filmmakers) and needed no training. They fell for a minute or two before their parachutes opened and they floated gently downwards for another five to make a soft sliding landing in the grass. Easy peasy. Would I do it again? NO.

See it here:

Mad Jump

Subversion and Incompetence – the Chilcot Inquiry

When Tony Blair’s embarrassingly titled memoir A Journey was published some years ago I was one of those who was delighted to find it in the Crime section of my favourite bookshops. The most ardent Stop the War campaigners would remove it from Biography or Politics and put it where it truly belonged.

blair

I was against the Iraq War. I’d read a number of books by weapons inspectors and was convinced, as were most of us, that the weapons of mass destruction claims, and the Al Quaida links claims, were nonsense. It also seemed very likely that whatever might follow would be at least as bad as Saddam Hussein’s regime. I therefore looked forward to the Chilcot Inquiry and was an avid spectator as the great and the good were hauled over the coals (sometimes rather gently) by John Chilcot and his team, which included the great historian Martin Gilbert, who, sadly, died before the Inquiry completed its work.

What was astonishing, though unsurprising, was how eloquently, articulately and unashamedly Tony Blair, and his colleagues, defended the Government and its decisions. I saw Blair perform at the Leveson Inquiry into Press Freedom too, and was equally impressed. The man was utterly convinced of his rightness. What is it that enables Great Leaders to fend off criticism so smoothly?

The answer is Zeal – than which there is nothing more dangerous in the world (putting aside the forces of nature).

The zealot rises above and beyond the evidence to promote his or her brand of faith, whether religious or political, or both. Righteousness and moral certainty, often impervious to evidence and doubt, are as dangerous as any weapons of mass destruction. Zealots live in another world. Indeed, so elevated is Blair’s state of mind that he is unaware that ‘It’s a journey’ is a cliché or that LOL means Laugh Out Loud, not Lots of Love. He has no sense of the ordinary.

There are three great political events this year which I look forward to with huge excitement – the EU Referendum, the US Presidential Election, and the publication, on July 6^th of the Chilcot Inquiry Report. I sympathise with those who castigate John Chilcot for his tardiness, but it is much more important that the report be exhaustive, comprehensive, and invulnerable to accusations of sloppiness, than that it should be rushed into print. The report will run to two and a half million words. That’s a modest tome after seven years of work.. It would only take me around four additional years to blog the same number of words (650 words daily every day of the year).

Like many others, I have feared that the mandarins would preside over a whitewash, but if yesterday’s leaks are anything to go by, the Inquiry will deliver stinging rebukes to Tony Blair, Jack Straw (former Foreign Secretary) and Richard Dearlove (former head of MI6) in particular. The first for subverting Cabinet Government and going to war with buddy George without good cause, the second for Straw’s incompetent management of the aftermath, and the last for Dearlove’s failure to stop Blair’s Government from publishing the highly misleading dodgy dossier, which made the case for war on the basis of unsubstantiated claims that Saddam Hussein possessed weapons of mass destruction and could use them against Britain in 45 minutes. It’s probably Blair’s subversion that has rankled most with the mandarins. The Butler report also took him to task for his presidential style of government.

Let’s hope the leaks are not themselves misleading. If they are not Mr Blair, the worst part of your journey has yet to come.

Making America Great Again

There’s a sneaky video doing the rounds of Facebook – Not the Greatest – that purports to pour honest cold water over the idea that America is the Greatest Country in the World. Its caption is ‘The Most Honest Three Minutes in Television History’ and it supposedly shows the response of an unnaturally passionate and eloquent panellist to the question ‘Can you say why America is the greatest country in the world’ posed naively by a young woman in the audience.

It’s produced and published by a media company called NextNextNow, and it’s obviously scripted. It’s entirely dishonest from the start. But spookily clever.

You think for a moment that the idea is to take down those stupid American Supremacists. The ‘panellist’ reels off statistics on prisoner numbers, child mortality, educational standards, exports and health, in order to prove that America isn’t the greatest except in some rather unattractive ways.

But watch carefully what happens next. The video morphs into a misty-eyed nostalgic account of a time when America truly was the greatest country in the world, the most innovative, the most moral. The implication, clearly, is that American can be made the greatest again with the right man in charge, perhaps.

Listen to what the panellist says:

‘We sure used to be (the greatest).’

‘We stood up for what was right. We fought for moral reasons. We made laws and struck down laws for moral reasons. We waged war on poverty not poor people.’

Now hang on a moment!

Vietnam, Cambodia, Central America, Iraq, capital punishment. And on the positive side, health care reform, gay marriage, etc.

When was this golden age when America was the shining beacon of morality?

It’s a clever video, never more so than when the man says:

‘We never identified ourselves by who we voted for in the last election.’

Which means, of course, that just because you voted for Obama last time you mustn’t think that you can’t vote for Donald this time round.

It’s also subtly anti=establishment. It’s designed to appeal to the cynical and to the angry. The message is that clichés and received wisdom must be rejected, together with all the establishment assumptions. Most of all, don’t trust those in power. There used to be ‘great men’ but there aren’t any great men any more to whom you can turn.

It is clever, evil nonsense. And it’s a very sneaky ad from the Trump camp.

Don’t get taken in. The most honest three minutes? Pernicious nonsense!

There is no greatest country in the world. What does ‘greatest’ mean, anyway?

Not to be confused with Chechnya

‘I disagree with the name Czechia,’ Regional Development Minister Karla Slechtova tweeted some time ago. ‘I don’t want people to confuse our country with Chechnya.’

Confusion, in general, seems to surround the suggestion that we should now call the sententiously named Czech Republic, where I live, Czechia. As I understand from my Czech colleagues at LLP Group the attempt, spearheaded by the Czech Foreign Minister, to rename the country (or rather, the Republic) has failed. The Government didn’t approve the plan.

I myself thought it a rather good idea, and I started to use the term ‘Czechia’ in all the marketing texts I write and on every other occasion that arose. Apparently I even began to mutter the new name in my sleep. It’s simple and so much less ponderous than the longer form. And I really don’t think it’s likely to be confused with Chechnya (but then, as a resident of the place, I know that ‘ch’ must be pronounced as a ‘k’ so Czechia doesn’t sound in the least like Chechnya to me).

states

I wondered how countries get their names and how territorial nomenclature is governed. It isn’t. Confusion and disorder abounds. Look at the complete list of sovereign states on Wikipedia (not an uncontroversial list, since not all of them recognise each other). Most sensible states are designated using one-word nouns or proper names (Austria, Afghanistan, etc.). And many of them tell you something about where they are (African, for example), where they are in relation to one another (North or South Korea), the kind of political entity they are (Republic, Kingdom, Emirate, etc.), or the physical feature from which they are formed (Islands). Most also imply the kind of people you might find in the territory (Hungary, Albania, etc.).

Only a small minority take the form of adjective followed by the type of political entity, such as:

Central African Republic
Czech Republic
Dominican Republic
United Arab Emirates
United Kingdom
United States
Vatican City

Only ‘Czech’, ‘Dominican’ and ‘Arab’ give you any clue as to the kind of people you might find in their territories (does Vatican tell you that they all wear frocks?) and ‘Dominicans’ from the Dominican Republic only use the sententious form to distinguish their state from Dominica.

As for the United Kingdom and the United States, those names tell you nothing beyond the nature of the political entity (we are subjects in the former, and they are citizens in the latter).

One or two do it another way round:

Democratic Republic of the Congo
Republic of Congo
Federated States of Micronesia

Just a few are based on their physical form:

Marshall Islands
Solomon Islands

These might well consider calling themselves the Marshalls or the Solomons.

But the Netherlands are no longer the Nether Lands.

Wouldn’t it be entirely sensible, and easier for students of geography, if the United Nations reviewed the names of states and prescribed a standard form such as:

The [continental] [optional physical nature] [political nature] of [majority people, or, where none, other term] and a formula for deriving a short form.

So, for example:

The European Republic of Moldova
The European Kingdom of Belgium
The European Republic of Czechia
The European Island Kingdom of Britain
The Caribbean Island Republic of Trinidad and Tobago
The Eurasian Dictatorship of Russia

Sententious, but informative and practical

Short forms:

ERMOLD
EKBELG
ERCZECH
EIKBRIT
CAIRTT
EDRUSS

Though I’m not sure these are catchy enough.

Fixed Price and Agility

I’ve come to love ‘agile’ systems implementation methods, and they are rightly fashionable in the nerdy circles in which I move.

But when I was trained on business software implementation methods many decades ago, I was told that it had to go like this:

find out what the customer wants, and write it all down,
get his sign-off on the requirements document,
design the system, and write it all down,
get his sign-off on the system specification,
build the system,
get his sign-off on the system,
and so on.

It didn’t work well. In the first step you might get it nearly 95% right, though there was still a danger that the customer didn’t know what he wanted until he’d seen what he might get.

But the second step was usually the dangerous one. If you’d gained the customer’s trust by the time you’d put the specification in front of him, he’d usually sign it off, even if he had no idea what it meant. System specifications are usually written in an inhuman language using the terminology of the target system. Crucially, they convey nothing at all about what it might be like actually to use a system.

The system build would often happen somewhere far away in a software laboratory, to which, of course, the customer would usually be denied access.

And then the last step didn’t happen, because when he saw what you’d built he wasn’t usually very happy.

agilecascade

This was the old-fashioned ‘cascade’ method and it’s rightly out of fashion. The ‘agile’ method involves more frequent meetings, discussions and demonstrations, so that the customer sees what you’ve understood, what you’ve designed, and what you’ve built and can quickly tell you if you’ve got it wrong before it’s too expensive to change course. When you’re implementing highly configurable software systems such as our time@work, expense@work or forms@work products, or Infor’s SunSystems, prototyping is relatively easy and this method is perfect.

At systems@work we’re doing a big implementation project for a fund services provider. It involves expenses, vendor invoice management, billing, timesheets and lots of workflow and reporting, and all the accounting that goes with all the transactions that are implied. Yesterday we conducted an important prototyping session, and I’m happy to say that it went well. There are a few things that we must change, but there are no fundamental misunderstandings.

But what about the agile method when a project is a fixed price one? Isn’t it potentially a recipe for disaster, in that changes of course are unpredictable and there’s no single moment when you can pin down the project scope in a system specification and treat all subsequent changes as separately chargeable?

This is not an easy question to answer. Wise and experienced project managers will build in a large ‘contingency’ component and perhaps put more emphasis on getting the requirements document right (even this stage of the project can involve proof-of-concept prototype demonstrations). But if you allow changes at later stages it’s hard to charge extra for them.

There is no perfect method. I would only say that if you believe the old ‘cascade’ method is a better one for large fixed price projects you are probably deluding yourself. You may be able more easily to define what’s out of scope, but it will be no easier to obtain the customer’s agreement that he should pay for all the ‘mistakes’ and scope creep, whether they are demonstrably his fault or yours.

Adam Bager

Occasional thoughts on IT, Business and Other Matters

Monthly Archives: May 2016