There has been controversy around p-values in recent years, often linked to issues with reproducibility in psychology.  p-values are also often reported in empirical software engineering papers. We haven’t yet seen widespread public controversy about software engineering studies, but that’s not because there aren’t problems!
The American Statistical Association has just released a clarifying statement about p-values. (pdf)
p-values are not inherently broken. The problems are about mis-interpreting them, about poor study design and practice, and about poor reporting. The ASA statement seems like a useful contribution to the debate.

What is engineering? Sometimes people think engineering is just the same as science, but in a new paper on the philosophy of engineering (preprint here), I argue why that’s not the case. Engineering is similar, but different to Science, and its epistemological issues are also similar but different.
I got into this question because of problems in assurance for software engineering and formal methods that are essentially philosophical problems. But having work available on the philosophy of engineering available should also help with perennial questions like “Is Software Engineering a field of engineering?” and “Is Computer Science a science?”.

Software Engineering would be a more mature discipline if we had spent more time reading What Engineers Know and How They Know It: Analytical Studies from Aeronautical History rather than A Pattern Language: Towns, Buildings, Construction.

Marek Kowalkiewicz from the SAP Research in Brisbane just last week won the international “Demo Jam” competition in the SAP TechEd event in LA, for the “Innoboard” software.  Innoboard is an augmented reality technology, which lets distributed teams interactively share whiteboards that mix projected images and physical sticky post-it notes.  All using the low-cost iphone camera and an ordinary projector.  Cool demo!  The idea at the end of taking streamed information out of the interactive session and using that to drive other workflow software (Jira in this case) is also cool, and just hints at the huge potential of ideas like this.
The Innoboard team found its first industry trial partner through the Future Logistics Living Lab, which is run by NICTA, SAP, and Fraunhofer IESE, and has around twenty (and growing) industry & research participants.  (Fraunhofer’s involvement is through the Fraunhofer Project Centre in Transport and Logistics at NICTA).  Industry trials for Innoboard are continuing, in a use-case for distributed logistics operations planning.  The Future Logistics Living Lab is also hosting a demo instance of Innoboard, and setting it up in the lab has helped contribute to ironing out some of the use & set-up issues in the early prototypes.
 

Robin Milner gave a presentation “Is Informatics a Science?” at a conference at ENS, 10 December 2007, where he discussed the challenge of better understanding relationships between models in computer science – how they “explain” (specify, refine, implement, abstract, realise) each other. I don’t believe he captured these thoughts in a journal or conference paper, but the ENS presentation follows an earlier similar 2006 presentation (for which there is a transcript) on “Scientific Foundation for Global Computing” .
An audio recording of the ENS presentation exists.  I’ve created a PDF transcript of that recording.  However, I don’t have the slides that Robin presented – I’d be interested to have a copy if anyone could send me one.

What is software architecture? There have been many definitions. Here’s mine.
First let’s consider some of the earlier definitions. SEI has a huge collection of definitions on its website, including “classic” definitions, bibliographic definitions (stops in 1996?), “modern” definitions, and definitions submitted from the community.  Perry and Wolf (1992) have perhaps the most classic definition, though it’s a little sketchy:

Architecture = {elements, form, rationale}

where elements are Processing, Data, or Connecting elements. Taylor et al. (2010) note that when people talk about software architecture in terms of Components and Connectors, that’s an over-simplification of Perry and Wolf’s definition – over-simplified because it doesn’t always work. For example in REST, Data elements are pre-eminent.
The ANSI/IEEE 1471-2000 definition expands on Perry and Wolf’s definition, and also slips environment into the scope of form (relationships).

Architecture is the fundamental organisation of a system, embodied in its components, their relationships to each other and the environment, and the principles governing its design and evolution.

Elements (components) and their form (relationships) are clearly key to understanding what software architecture is, but many people think that’s all it is! Instead, software architecture researchers have understood for a long time that rationale (design/evolution principles) is also a key part of what software architecture is.
But contrari-wise, software architecture is not just rationale. So although Taylor et al. (2010) is a great architecture textbook, their definition of software architecture isn’t so great:

A software system’s architecture is the set of principal design decisions made about the system.

I would instead say that design decisions are the means by which elements, form, and rationale are created. The design decisions are not the architecture per se.
Software architecture is commonly misunderstood to be an exclusively structural model. Perhaps that’s because UML class diagrams and deployment diagrams are often presented as iconic for software architecture. The definition from the Bass et al. (2008) classic textbook also encourages this view:

The software architecture of a program or computing system is the structure or structures of the system, which comprise software elements, the externally visible properties of those elements, and the relationships among them.

But there has been a shift in the software architecture research community to think about architecture more abstractly: in terms of rules or constraints or styles. So instead of “structures”, I prefer using the word “abstractions”, to more easily accommodate a rule-based architectural perspective. The abstractions here are not just of software, but also non-software elements in the system, including the environment. Fielding (2000) also talks about abstractions this way (though for some reason he limits software architecture to run-time, which I don’t agree with):

A software architecture is an abstraction of the run-time elements of a software system during some phase of its operation. A system may be composed of many levels of abstraction and many phases of operation, each with its own software architecture.

Apparently Clements et al. (2010) has another new definition:

The software architecture of a system is the set of structures needed to reason about the system, which comprise software elements, relations among them, and properties of both.

Here we have (ignoring “structures” for the moment) Perry and Wolf’s elements and form (relations) again, but now also with properties for each. Perry and Wolf’s rationale has not disappeared, but here appears as a qualifier (“needed to reason about”). I like this. I think the whole point of architecture is abstraction and analysis of a system for particular purposes. Here the “reasoning” objective implicitly encompasses those purposes and analyses.
I’d slightly prefer to say “communicate and reason” instead of “reason”, though perhaps you could say that understanding a communication is-or-requires reasoning. I’d also prefer to talk about “a” (not “the”) software architecture of a system (and similarly “a set” not “the set”), to more readily accommodate multiple views/perspectives of a single system.
So, in summary, my definition would be something like:

A software architecture of a system is a set of abstractions needed to communicate and reason about software in the system. A software architecture models elements of the system and its environment, relations among those elements, and properties of those elements and relations.

Just heard in a QESP webinar on Software Innovation in Australia from Julian Day of the Australia Consensus Awards:

In business, invention is the conversion of cash into ideas, but innovation is the conversion of ideas into cash.

Nice.  I see this is also on wikipedia.  I wonder what’s the original source for this quote?

Liming has raised three points in reference to my Fractal V Lifecycle.  His questions are probing the limits of the model in interesting ways. Before I discuss them, I’d like to introduce a concept and some terminology from an earlier paper I wrote.
The V model can accommodate as many levels of design abstraction as you like.  For example, if you do high-level design/integration testing, add a layer for that between system architecture/system testing and low-level design/component testing. At each level of design abstraction, you have the same schema of activities: design the artifact (called “Plan” in the paper), pass it on for elaboration to a lower level of abstraction (called “Do” in the paper) and then verify that the design and implementation are consistent (called “Check” in the paper).  There is also another kind of activity in the V model, “Plan-to-Check”, which can proceed concurrently with the “Do” step.
So, the left-hand side of the V is “Plan”, the horizontal dotted lines are “Plan-to-Check”, the pointy bit is “Do”, and the right-hand side is “Check”.
On to Liming’s points about situations quoted or paraphrased as follows…
1. “Left side of the V bends up before reaching the bottom”

Liming wonders – how can you “Check” a developed artifact if it hasn’t been built yet!?  As Liming suggests, I want to argue that if you don’t have an implementation yet, you of course can’t test it, but you can certainly analyse the plan/design.  Your “Check”ing activity will be a requirements review, or a model-based design simulation, or perhaps testing with stubs (or “mock objects” if you prefer).  It depends on your level of design abstraction and what artifacts you already have from any previous iterations.

Liming thinks analysis is implicit in the left-hand-side of the V, because designers always carry out some sort of analysis immediately after creating artifacts.  I agree designers often do that sort of analysis, but I disagree that it’s implicit in the left-hand-side of the V!  I think it’s simpler and more consistent to think about it as being part of the right-hand-side of the V, with a null “Do” activity.  So designers would do many mini design/analyse (“Plan”/”Check”) iterations at their level of design abstraction before sending it off to the lower level of abstraction for elaboration.

2. “Right side of the V dips down before reaching the top”

Liming observes that this isn’t just about avoiding end-customer contact.  I didn’t highlight that in my original blog article, but I fully agree.  For example, you might have unit tested iterations that you don’t send up for integration testing just yet. These are “internal iterations” at some level of deign abstraction, avoiding the “customer” at the next highest level of design abstraction.

3. “Early or continuous testing breaks the V”

Liming claims that as testing happens on the right-hand side of the V, early or continuous testing means you can’t have a V any more.  I agree early and continuous testing breaks the V, but it doesn’t break the Fractal V!  In the Fractal V, yes you do still carry out the testing on the right-hand side of the V, but the whole point of the Fractal V is that it can accommodate a variety of different kinds of internal iterations.

I’m not claiming that all software development will fit the Fractal V Lifecycle.  For example, imagine a lifecycle with long-running requirements analysis performed concurrently with ongoing development, where you could pre-emptively abort a development iteration depending on the intermediate outcomes of that ongoing requirements analysis.  I don’t know of any lifecycle that would capture that exactly.  Maybe this is Liming’s point about continuous integration – the hairy reality is that developers won’t stop entirely while they’re waiting for feedback from continuous integration – they’ll probably already have started the next unit of work.  I would rebut that the aim of continuous integration is to given “immediate” unit and integration test feedback to developers – to let them do many micro-iterations as quickly as possible.  I say you should probably just work around that hairy reality for the sake of simplicity.  No model is perfect, but some models are still useful.

Lifecycle models are used because they make it easier for large groups to understand and coordinate complex projects.  The Fractal V provides more flexibility for iteration than the normal V lifecycle, but is still reasonably simple, and importantly retains the Plan/Do/Check schema that so many systems engineering disciplines and tools rely on.
It will be interesting to see the un-V lifecycle Liming mentions (I hope they find a better name!), and see how it deals with all of these issues.

Louis has got a new article version of my earlier blog post on the Fractal V Lifecycle up on alinement.net magazine/community website. He’s also added some thoughts of his own on extending the concept into pre-project activities.  For these sorts of activities I tend to favor putting them on top, i.e. on the left but at a higher level – giving you a taller V.  But there’s no hard and fast rule… use it whatever way helps!

NICTA’s recent Techfest in Sydney saw a flurry of news around the announcement of a significant research achievement- the formal verification of the seL4 microkernel. The team developed a mathematical proof of the functional correctness of the microkernel down to the level of the C source code.
The achievement is important for two reasons. Firstly, it makes it possible to prove code-level functional correctness of whole computer systems based on the L4 microkernel. This means computer systems can carry a new kind of assurance, supporting strong arguments for safety and security for high-integrity software systems.
Secondly, it makes the creation of these proofs more feasible in practice. It should now be possible to formally verify properties of whole systems where only the key parts of the system are formally verified. This is critical for the practical application of formal verification. The verification of the L4 microkernel took more than 20 person years of effort to verify just 7500 lines of C source code. Modern embedded computer systems can have millions of lines of source – it’s not practical to formally verify all the code in such systems at these levels of productivity.
However, you don’t need to verify all the code! L4 provides rigorous separation between processes running in the microkernel, and that separation is guaranteed by the recent proof. If you can isolate the safety-critical or security-critical parts of an entire system to one small formally verified component running in an L4 process, it should be possible to lift the guarantees for that component to the whole system, even if the other components in the system haven’t been verified.
That is the challenge for a new project at NICTA – Trustworthy Embedded Systems. The plan is to develop technologies to support the creation and verification of entire systems running on top of the microkernel. This is a large project involving several NICTA labs and researchers from many disciplines (operating systems, formal methods, software architecture). I’m part-allocated to the project for the next few years. My PhD was in formal methods, and this is really the first time I’ve dipped my toe back into that area for the last decade. However, I won’t be doing too much theorem proving myself – my focus in the project will instead largely be on other software engineering issues in this context, such as configuration management, and how to use the component architecture to support product line development.